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This  thesis  deals  with  the  problem  of  unifying  in  a 
single  formalism  the  specification  of  data  and  algorithms. 
A  conventional  program,  on  the  one  hand,  specifies  a 
computational  algorithm,  but  does  not  explicitly  describe 
the  data  processed.  The  set  of  accepted  inputs  and  their 
structure  must  be  inferred,  if  possible,  from  the  program. 
A  grammar,  on  the  other  hand,  specifies  the  elements  and 
structure  of  a  language.  While  the  elements  of  such  a 
language  may  serve  as  the  input  data  to  a  program,  a  grammar 
does  not  describe  how  to  process  that  data.  This  thesis 
develops  a  formalism  that  synthesizes  the  two,  specifying  in 
a  unified  manner  the  precise  set  of  inputs  accepted,  how 
they  are  structured,  and  how  they  are  processed.  The  use 
and  value  of  the  formalism  are  demonstrated. 
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CHAPTER  1  -  INTRODUCTION 


Conventionally,  a  programming  language  is  used  to 
specify  a  computational  algorithm,  and  a  grammar  is  used  to 
specify  the  structure  of  data.  Neither  of  these  formalisms 
reflect  the  strong  mutual  dependence  that  data  and 
algorithms  that  process  the  data  have  on  each  other  in  a 
given  problem.  In  this  thesis  we  develop  a  formalism  that 
synthesizes  the  specification  of  algorithms  and  data. 

Since  grammars  occupy  such  a  central  position  in  this 
thesis,  we  devote  Chapter  2  to  a  presentation  of  grammatical 
preliminaries . 

The  need  for  a  synthesis  is  motivated  from  a 
programming  point  of  view  in  Chapter  3  by  presenting  and 
appraising  the  program  design  method  developed  by  Jackson 
[75].  The  Jackson  Method  is  based  on  the  premise  that  a 
program  structure  should  mirror  the  structure  of  the  data  it 
processes.  Jackson* s  approach  is  very  attractive  and  serves 
as  the  basis  for  our  own  design  efforts.  There  are, 
however,  problems  with  the  way  the  design  approach  is 
developed  into  a  design  method.  Most  of  these  problems  stem 
from  the  fact  that  a  program  must  somehow  determine  for 
itself  the  structure  of  the  input  data,  whereas  in  a  grammar 
the  structure  is  specified  directly.  A  grammatical 
formalism  provides  an  elegant  separation  of  design-oriented 
from  implementation-oriented  concerns. 
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CHAPTER  2  -  GRAMMARS 


In  this  chapter  we  present  the  basic  notations  and 
formalisms  that  are  used  throughout  the  thesis.  Much  of  the 
chapter  is  a  review  of  formal  language  theory  and  can  be 
found  in  [DeRemer  74].  Here  we  are  concerned  with  the 
syntactic  aspects  of  language,  and  take  context-free 
grammars  as  our  starting  point.  A  more  in-depth  presentation 
can  be  found  in  [  Aho  8  Oilman  72]. 

2 . 1  Preliminaries 

We  denote  the  ent£tz  (or  anil)  string.  the  string  with 
no  symbols,  by  the  symbol  e.  It  is  the  identity  element  for 
strings  under  the  concatenation  operation.  If  X  and  Y  are 
sets  of  symbols,  then  XY,  called  the  product  of  X  and  Y,  is 
defined  as: 

XY  =  {xy  |  x  6  X  and  y  6  Y) 

If  X  is  a  set  of  symbols,  then 

(1)  X  power  0  =  {e} 

(2)  X  power  n+1  =  X (X  power  n)  for  n>0 

(3)  X*  =  Onion  of  X  power  n,  for  all  n>0 

(4)  X+  =  Union  of  X  power  n,  for  all  n>1 

X*  is  called  the  closure  of  X,  and  X+  is  called  the  positive 
closure  of  X.  Intuitively,  X*  is  the  set  of  all  finite- 
length  strings  formed  from  symbols  in  X,  including  the  empty 
string.  A  language  L  is  a  possibly  infinite  set  of  finite- 
length  strings  over  a  finite  set  of  symbols  (or  alphabet) . 
The  operations  defined  above  apply  also  to  languages. 
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2.2  Context-Free  Grammars 


A  grammar  is  a  finite  specification  of  a  possibly 
infinite  language.  We  shall  consider  only  a  particular, 
mathematical  form  of  grammar.  Strings  in  the  language 

generated  by  a  given  grammar  are  produced  by  starting  with  a 
string  consisting  of  the  particular  symbol  called  the  start 
symbol  and  successively  transforming  the  string  by  applying 
rewriting  rules. 

Formally,  a  context-free  grammar  (cfg)  is  a  4-tuple 
G= (Vn,Vt,R,S)  where 

(1)  Vn  is  a  finite  set  of  symbols  called  nonterminals. 

(2)  Vt  is  a  finite  set  of  symbols  called  terminals  such 

that  Vn  and  vt  are  disjoint.  We  sometimes  enclose 
terminal  symbols  in  quotes  to  distinguish  them 

visually . 

(3)  R ,  the  set  of  grammar  rules,  is  a  finite  subset  of 

Vn  x  (Vn  U  Vt)*.  Each  grammar  rule  (A, a)  in  R  is 

written  A:  a. 

(4)  S  is  a  distinguished  member  of  Vn  called  the  start 
syjnbol  or  goal. 

By  convention  we  use  capital  letters  (A,B,C,...)  to 
denote  nonterminals,  lower  case  letters  at  the  beginning  of 
the  alphabet  (a,b,c,...)  to  denote  terminals,  lower  case 
letters  at  the  end  of  the  alphabet  (...,x,y,z)  to  denote 
strings  of  terminals,  and  lower  case  underlined  letters 
(a,b,c,...)  to  denote  strings  over  V  =  Vn  n  vt. 

A:  a  1  |  a2  |  ...  1  an  is  a  notational  shorthand 
denoting  the  set  of  rules  A:  al ,  A:  a2,  ...  ,  A:  an. 

The  relation  =>,  read  as  "directly  (or  immediately ) 
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derives11,  is  defined  as  follows:  If  aBc  is  a  string  in  V* 
and  B:  b  is  a  grammar  rule,  then  aBc=>abc.  We  call  aBc=>abc 
a  direct  (or  immediate)  der ivat i on.  We  use  =>+  and  =  >*  to 
denote,  respectively,  the  transitive  closure  and  reflexive 
transitive  closure  of  =  >.  =>♦  is  read  as  "derives  in  a 
nontrivial  way"  and  =>*  as  "derives".  Wore  intuitively, 
a=>*b  if  there  exists  a  sequence  of  strings  aO ,  al,  ...  , 
an  where  n>0  such  that  a  =  aO ,  ai  =  >  a(i+1)  for  0<i<n-1,  and 
an  =  b.  If  n>1,  then  a=>+b .  Trivially,  a=>*a  for  all  a  in 
V*. 

A  sentential  form  is  a  string  derivable  from  the  start 
symbol  S,  i.e.  an  a  such  that  S=>*a.  A  sentential  form 
containing  no  nonterminals  is  called  a  sentence.  The 
language  L1G)_  generated  by  grammar  G  is  the  set  of  all 
sentences  for  that  grammar,  i.e.,  L(G)  =  {x  6  Vt*  |  S=>+x} . 

A  derivation  can  be  displayed  graphically  with  a 
derivation  tree.  In  a  derivation  tree,  each  nonterminal 
used  in  the  derivation  labels  a  subtree  root  node  whose 
direct  descendants  are  labelled  with  the  symbols  on  the 
right  side  of  the  grammar  rule  used  to  expand  the 
nonterminal.  Thus,  the  root  node  is  labelled  with  the  start 
symbol  S,  and  each  leaf  is  labelled  with  a  terminal. 

For  a  given  sentential  form,  derivation  trees  are 
unique,  but  derivations  are  not.  We  shall  distinguish  two 
particular  derivations.  A  leftmost  derivation,  indicated  by 
=>*lm,  is  one  in  which  at  each  step  the  leftmost  nonterminal 
is  rewritten.  A  rightmost  derivation  is  defined  similarly. 
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substituting  "rightmost”  for  "leftmost"  and  "rm"  for  "lm"  in 
the  above  definition. 

A  grammar  G  is  said  to  be  ambiguous  if  there  exists  at 
least  one  sentence  in  L (G)  for  which  there  is  more  than  one 
derivation  tree,  or  equivalently,  more  than  one  leftmost 
(rightmost)  derivation. 

We  consider  a  derivation  to  be  a  passive  relation 
between  strings.  It  is  possible  to  take  an  active  view  of 
"a  =>  b".  If  we  obtain  b  from  a,  we  say  that  a  directly 
prod uces  b  or  that  b  is  a  direct  production  of  a.  A 
production  is  a  nonempty  sequence  of  direct  productions, 
i.e.,  a  produces  b  if  a  =>♦  b.  Similarly,  if  we  obtain  a 
from  b ,  we  say  that  b  directly,  reduces  to  a  or  that  a  is  a 
direct  reduction  of  b.  A  reduction  is  a  nonempty  sequence  of 
direct  reductions,  i.e.,  b  reduces  to  a  if  a  =>♦  b. 

We  say  a  nonterminal  A  is  left-recursive  if  A=>  +  Aa  for 
some  a.  Similarly,  A  is  right- recursive  if  A=>+bA  for  some 
b. 

An  example  of  a  cfg  is  G=  (Vn  ,Vt,  R,Exp)  where 

Vn  =  {Exp, Term, Factor , Addop,  Mulop} 

Vt  —  {(/)#*■/“#*#/#&}  and 

R  contains  the  grammar  rules: 

Exp  :  Exp  Addop  Term  |  Term 

Term  :  Term  Mulop  Factor  |  Factor 

Factor  :  •(•  Exp  •)•  |  'a' 

Addop  :  |  •-* 

Mulop  :  |  */' 

L (G)  is  the  set  of  arithmetic  expressions  that  can  be  formed 
using  -,  *,  and  /  as  operators,  "a"  as  an  operand,  and 
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parentheses  to  bracket  subexpressions.  The  grammar  will 
structure  an  expression  according  to  the  usual  arithmetic 
rules  of  precedence  and  associativity. 


2 . 3  Parsing  Cfg • s 


Given 

a  terminal  string  x  and  a  cfg 

G,  we  want  to 

know 

if  x 

is  in 

L  (G)  or  not. 

In  practice,  we 

need  to  know 

more 

than 

just  » 

x  6  L  (G)  ?". 

We  also  need  to 

know  the  structure 

of  X 

,  as 

determined  by 

its  derivation 

tree,  because 

its 

structure  helps  determine  its  meaning.  Algorithms  that 
answer  these  questions  are  called  par si ng  algorithms. 

A  parsing  algorithm  is  categorized  in  each  of  three 
separate  areas.  The  first  area  is  vertical  strategy:  the 
parser  may  start  with  the  distinguished  nonterminal  and  work 
down  the  derivation  tree  to  the  terminal  string  (tog-down)  , 
it  may  start  with  the  terminal  string  and  work  up  the 
derivation  to  the  distinguished  nonterminal  (bottom-u  2)  r  or 
it  may  mix  the  two.  Top-down  methods  typically  construct  a 
leftmost  derivation  while  bottom-up  methods  typically 
construct  a  rightmost  derivation.  The  second  area  is 
horizontal  strategy:  the  parser  may  work  from  left  to  right, 
right  to  left,  or  alternate  between  the  two.  All  algorithms 
we  will  consider  work  from  left  to  right.  In  the  third 
area,  we  categorize  the  algorithm  as  deterministic  or 
nondeterministic.  Nondeter minis tic  parsers  involve  guessing 
and  must  backtrack  if  a  guess  turns  out  incorrect. 
Nondeterministic  parsers  sacrifice  efficiency  for  power. 
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They  can  handle  all  cfg's  while  deterministic  parsers  can 
handle  only  a  proper  subset.  But  nonde terministic  parsers 
are  much  less  efficient  in  terms  of  both  time  and  space.  In 
practice,  we  are  very  concerned  that  our  grammar  be  parsable 
efficiently,  therefore  deterministically. 

Of  the  numerous  deterministic  parsing  algorithms,  the 
most  appropriate  for  the  types  of  grammars  we  later  consider 
are  the  so-called  LR  and  LL  techniques. 

2.3.1  LR  Parsing 

LR  parsers,  introduced  in  [  Knuth  65  ],  are  the  ultimate 
in  deterministic  techniques,  in  the  sense  that  the  class  of 
languages  parsable  by  LR  techniques  is  precisely  the  class 
of  deterministic  languages.  The  actual  LR  parsing  algorithm 
is  not  of  interest  here  and  as  such  is  not  examined.  The 
interested  reader  should  consult  such  excellent  references 
as  [  Aho  6  Johnson  74],  [Horning  74],  [  Aho  Z  Oilman  72]  and 
[  Aho  S  Oilman  73],  Rather,  we  will  be  principally  concerned 
with  some  grammatical  properties  of  the  particular  subclass 
of  LR  grammars  known  as  LR  (k)  grammars. 

Before  giving  the  LR  (k)  definition  we  need  some 
preliminary  definitions.  If  S  =>*rm  aAw  =>rm  abw  =>*rra  xw 
is  a  rightmost  derivation,  then  we  call  b  a  handle  of  abw. 
That  is,  a  handle  of  a  right-sentential  form  is  a  substring 
which  is  the  right  side  of  a  grammar  rule  and  which  can  be 
replaced  by  the  left  side  of  that  rule  so  that  the  resulting 
string  is  also  a  right-sentential  form.  We  define: 
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FIRST_k(a)  =  (w  €  vt*  |  either  |w|<k  and  a=>*w 

or  |w|  =  k  and  a=>*wx  for  some  x} 

Informally,  FIRST_k(a)  contains  the  terminal  strings  of 

length  less  than  k  that  can  be  derived  from  a,  plus  the 

terminal  strings  of  length  k  that  prefix  strings  derivable 

from  a.  Finally,  the  LR (k)  definition  is  given  in  terms  of 

the  augmented  grammar  G*  which  ensures  that  the  start  symbol 

does  not  occur  on  the  right  side  of  any  rule.  That  is, 

G*  =  ({Vn  0  S'} ,  Vt ,  R  U  {S • ; S) , S •  )  such  that  S’  is  not  in  Vn. 

Informally,  we  say  that  a  grammar  is  LR  (k)  if  given  a 

rightmost  derivation 

S'  =  aO  =  >rm  a  1  =>rm  ...  =>rm  an  =  w 

we  can  isolate  the  handle  of  each  right-sentential  form  and 

determine  which  nonterminal  is  to  replace  the  handle  by 

scanning  ai  from  left  to  right  but  only  going  at  most  k 

symbols  past  the  right  end  of  the  handle  of  ai. 

An  LR  (k)  grammar  is  formally  defined  as  follows  [ Aho  5 

Oilman  72  ]: 

Let  G  be  a  cfg  and  G*  its  augmented  grammar.  We  say  G  is 
LR(k),  for  some  fixed  k  >  0,  if  the  following  three 
conditions 

(1)  S*  =>*rm  aAw  =>rm  abw 

(2)  S*  =>*rm  cBx  =>rm  aby ,  and 

(3)  FIRST_k(w)  =  FIRST~k(y) 

imply  that  aAy  =  cBx  (i.e.  a  =  c,  A -B ,  x=y)  . 

An  important  result  is  that  any  language  described  by 
an  LR (k)  grammar,  k>0 ,  can  be  described  by  an  LR(1)  grammar. 
Thus,  we  can  describe  the  entire  class  of  deterministic 
languages  with  just  LR(1)  grammars  [Aho  5  Oilman  72]. 
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2.3.2  LL  parsing 


It  has  been  shown  that  the  classes  of  LL  (k)  languages 
are  distinct  for  every  k,  and  are  properly  contained  in  the 
class  of  deterministic  languages.  Furthermore,  any  LL  ( k ) 
grammar  is  LR  (k)  . 

The  following  is  actually  a  theorem  '  in 
[ Aho  Z  Ullman  72],  but  is  presented  here  as  the  LL (k) 
definition  because  it  conveys  more  clearly  the  LL  condition. 
A  grammar  G=  (Vn, Vt,R , S)  is  LL(k),  for  some  fixed  k,  if  and 
only  if  the  following  condition  holds: 

If  A:  b  and  A:  c  are  distinct  rules  in  R, 

then  FIRST_k (ba)  Intersect  FIRST_k(ca)  =  0 
for  all  wAa  such  that  S  =>*lm  wAa. 

Informally,  the  LL(k)  definition  says  that,  given  a  left- 
sentential  form  wAa,  we  know  uniquely  which  A-rule  to  choose 
solely  on  the  basis  of  w  and  the  next  k  input  symbols 
following  w. 

As  a  result,  the  LL(k)  parsing  strategy  is  conceptually 
quite  simple:  it  is  basically  a  preorder  tree  traversal  of 
the  derivation  tree,  using  the  k-symbol  lookahead  string  to 
choose  among  possible  alternatives.  The  two  standard  ways 
to  implement  LL  parsers,  recursive  descent  and  table-driven, 
are  in  fact  similar  to  recursive  and  non-recursive  preorder 
tree  traversal  algorithms,  respectively.  with  recursive 
descent,  one  basically  transcribes  the  grammar  into  a 
program,  with  a  procedure  for  each  nonterminal.  With  a 
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little  practice,  one  can  write  recursive  descent  parsers 
very  quickly.  However,  because  of  the  current  inefficient 
implementations  of  (possibly  recursive)  procedure  calls, 
recursive  descent  parsers  generally  execute  slowly.  The 
table-driven  parser  uses  the  same  strategy  as  recursive 
descent  but  replaces  procedure  calls  and  recursion  with 
explicit  stack  management.  As  a  result,  it  executes  more 
efficiently,  and  is  usually  chosen  over  recursive  descent  in 
production  systems  (though  it  should  be  mentioned  that  the 
Burroughs  Extended  Algol  compiler,  written  for  a  machine 
designed  to  perform  procedure  calls  efficiently,  is  written 
using  recursive  descent) . 

From  the  above  LL(k)  definition,  it  can  be  seen  that  a 

grammar  with  a  left  recursive  nonterminal  is  not  LL(k)  for 

any  k.  Many  "natural"  grammars,  however,  do  contain  left- 

recursive  nonterminals,  and  must  be  transformed  if  we  wish 

to  use  an  LL(k)  parser.  For  example,  the  expression  grammar 

given  above  is  LR(1)  but  not  LL(k)  for  any  k.  It  can  be 

transformed  using  standard  techniques  to  an  LL(1)  grammar 

with  the  following  rules: 

Exp  :  Term  Termlist 

Termlist  :  Addop  Term  Termlist  j  a 

Term  :  Factor  Factorlist 

Factorlist  :  Mulop  Factor  Factorlist  |  e 
Factor  :  • (•  Exp  • )  •  |  'a' 

Addop  :  | 

Mulop  :  • * •  |  V 

2.4  Regularized  Context-Free  Grammars 

It  is  convenient  to  use  an  extended  form  of  context- 
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free  grammars  called  regularized  context-free  grammars 
(rcfg's).  In  an  rcfg,  each  grammar  rule  right  side  is  a 
regular  expression  over  the  terminal  and  nonterminal 
alphabet  V.  Regular  expressions  (RE's)  over  a  set  X  may  be 
defined  recursively  as: 

(1)  if  A  6  X  U  {e}  ,  then  A  is  an  RE; 

(2)  if  B  and  C  are  RE's,  then  so  are 

(a)  BC  (concatenation  -  "B  followed  by  C") , 

(b)  B  |  C  (alternation  -  "B  or  C")  ,  and 

(c)  B*  (repetition  -  "zero  or  more  B's")  . 

We  denote  the  set  of  RE's  over  X  by  RE  (X)  .  Except  where 
altered  by  parentheses,  the  precedence  of  RE  operators  in 
descending  order  of  binding  power  is:  repetition, 

concatenation,  and  alternation. 

The  set  of  instances  of  a  6  RE(X)  is  denoted  by  1(a) 
and  is  defined  as  follows: 

(a)  if  a  €  X  U  {e ) ,  then  1(a)  =  {a} 

(b)  if  a  =  b  c  and  b , c  e  RE(X),  then  1(a)  =  1(b)  1(c) 

(c)  if  a  =  b*  and  b  6  RE(X),  then  1(a)  =  (1(b))* 

(d)  if  a  =  bl  1  ...  1  bn  and  bi  6  RE  (X)  for  1<i<n,  then 

1(a)  =  Union  of  I  (bi)  for  1<i<n 

Note  that  for  all  a  6  RE (X)  ,  1(a)  is  a  subset  of  X*.  Also, 

if  a  e  X*,  then  1(a)  =  {a}.  An  instance  of  a  grammar  rule 

A:  a  is  a  rule  A:  a'  where  a'  is  an  instance  of  a. 

The  definition  of  a  derivation  ("=>")  is  extended  to 

rcfg's  by  specifying 

(al  |  a2  |  ...  |  an)  =>  ai  for  1<i<n 

a*  =>  (a  power  i)  for  i>0. 

where  " (a  power  i)  "  denotes  the  sequence  of  length  i 
"a  a  ...  a".  From  this  follows  the  definitions  for 
canonical  rcfg  derivations.  For  an  immediate  leftmost 
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derivation  we  have: 


x  (al  |  ...  1  an)  b  =>lm  x  ai  b  for  1<i<n 
x  (a)  *  b  =  >lm  x  (a  power  i)  b  for  i>0. 
where  x  6  Vt*  and  a,  ai,  b  e  RE(V).  Rightmost  derivations 
are  defined  analogously. 

Other  RE  operators  we  on  occasion  use  are:  +  ("one  or 
more  times") ,  *n  ("zero  to  n  times") ,  +n  ("one  to  n  times")  , 
n  ("n  times"),  and  ?  ("optional").  All  of  these  can  be 
obtained  by  compositions  of  the  basic  three  operators. 

Every  cfg  is  also  an  rcfg.  This,  together  with  the 
fact,  which  we  will  show  later,  that  an  rcfg  can  be 
transformed  into  a  language-equivalent  cfg,  means  that  the 
generative  power  of  rcfg’s  is  equivalent  to  that  of  standard 
cfg’s.  However,  in  terms  of  descriptive  power,  rfcg’s  are 
much  more  powerful.  For  example,  consider  the  rcfg  with 
rules: 

Exp  :  Term  ((’♦•  |  •-')  Term)  * 

Term  :  Factor  ((•*'  |  '/')  Factor)* 

Factor  :  ’  ('  Exp  ’)  ’  ]  *a* 

Of  the  three  expression  grammars  presented,  this  is  both  the 
clearest  and  the  most  concise. 

2 . 5  Parsing  Rcfg's 

Madsen  and  Kristensen  [75,  76]  extend  the  notion  of 

LB (k) -ness  to  rcfg's.  They  also  present  two  approaches  to 
parsing  rcfg’s.  In  the  first,  they  generalize  LR  parser 
theory  to  work  directly  on  rcfg’s.  In  the  second,  they 
define  a  straight orward  rcfg  to  cfg  transformation  that 
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preserves  LR  properties.  Madsen  and  Kristensen  conclude  that 
the  latter  approach  is  more  attractive  since  standard  LR 
parsing  techniques  can  be  used,  and  the  parsing  algorithm 
need  not  be  modified. 

Their  rcfg  to  cfg  transformation  is  as  follows.  Two 
auxiliary  transformations  are  used,  one  for  each  of  the 
operators  *  and  | .  These  transformations  replace  a  given 
grammar  rule  with  a  set  of  rules  which  simulate  its  effect. 
For  a  given  rule  in  G 

A:  a  (b)  *c  where  a,  b  and  c  are  in  RE(V) 

T*  (G)  defines  a  new  grammar  G*=(Vn  U  £C >  ,  Vt ,  R *,  S)  where  C  is 
not  in  Vn,  and  F*  is  the  same  as  R  except  that 

A:  a  (b)  *c  is  replaced  by  A:  aC,  C:  be,  and  C:  c. 

For  a  given  rule  in  G 

A:  a  (b 1  |  b2  j  ...  |  bn)  c  where 

a,  bi  (i=1,...,n),  c  are  in  RE(V) 

T  |  (G )  defines  a  new  grammar  G|  =  (Vn  0  {B  ,C}  ,  Vt  ,R  |  ,  S)  where 
B , C  are  not  in  Vn,  and  R|  is  the  same  as  R  except  that 
A:  a(b1  |  ...  |  bn)  c  is  replaced  by 

A:  aB,  B:  bic  (i=1,...,n),  and  C:  c. 

Successive  applications  of  T*  and  T|  to  an  rcfg  G  produce  a 
cfg  G'  =  T ( G) .  It  is  straightforward  to  show  that  L(G')  = 

L  (G)  .  Madsen  and  Kristensen  go  on  to  show  that  T  preserves 
LR (k)  properties. 

Before  we  formalize  the  definition,  we  can  see 
intuitively  what  it  would  mean  for  an  rcfg  to  be  called 
LL(k).  With  respect  to  the  repetition  operator,  it  would 
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after  having 


mean  that  for  a  given  rule  A:  a(b)*c, 
recognized  a  and  some  number  of  b*s  (zero  or  more) ,  we  must 
be  able  to  decide  unambiguously  on  the  basis  of  (at  most) 
the  next  k  input  symbols  whether  to  recognize  next  either  a 
b  or  a  c.  For  the  alternation  operator,  it  would  mean  that 
for  a  given  rule  A:  a(b1  \  ...  |  bn) c,  after  having 

recognized  a,  we  must  be  able  to  decide  unambiguously  on  the 
basis  of  (at  most)  the  next  k  input  symbols  which  bi  to 
recognize . 

Formally,  an  rcfg  G  is  rcfg-LL(k)  if  and  only  if  for 
all  a  in  RE (V)  such  that  a  occurs  in  a  grammar  rule  in  R, 
the  following  holds: 

(a)  a  =  bi  b2  ...  bn,  bi  in  RE(V)  1<i<n,  or 

(b)  a  =  (bi  |  ...  |  bn),  bi  in  RE(V)  1<i<n,  and 
FIRST_k  (bi  f)  Intersect  FIRST_k (b j  f)  =  0,  1<i,j<n, 
i*j,  for  all  x,f  such  that  S  =  >*lm  xaf,  or 

(c)  a  =  (b)*,  b  in  RE  (V)  ,  and 
FIRST_k(bf)  Intersect  FI RST_k  (f )  =  J2F, 
for  all  x,f  such  that  S  =>*lm  xaf 

where  FIRST_k  is  suitably  extended  to  a  domain  of  RE's. 

Since  the  transformation  T  preserves  the  FIRST_k  sets 
of  left-sentential  forms,  we  see  that  T  preserves  LL 
properties. 

We  would  expect  G  to  be  rcfg-LL(k)  if  it  is  cfg-LL(k). 
In  fact  this  is  true  since  all  rules  are  of  the  form 
A:  (al  |  ...  |  an),  n  >  1,  each  ai  is  in  V*  (a  subset  of 
R E ( V) ) ,  and  condition  (b)  reduces  to  the  cfg-LL  (k) 
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condition . 


2 . 6  Grammars  and  Structure 

It  is  important  to  recognize  that  a  grammar  imposes 
structure  on  sentences  in  the  languauge.  The  role  of 
nonterminals  is  to  abstract  structure. 

We  find  rcfg*s  more  attractive  precisely  because  they 
permit  us  to  specify  a  formal  phrase-structure  which 
coincides  with  the  viewed  logical  phra se- structure  of  the 
data.  The  higher-level  constructs  of  rcfg's  make  the  logical 
structure  more  transparent,  and  improve  both  readability 
and  wr iteability . 

The  fact  that  grammars  impose  structure  forms  the  basis 
for  the  entire  thesis.  The  specification  of  structure  is  a 
central  concern  in  problem-solving.  Rcfg’s,  as  we  have 
seen,  are  a  clear,  concise,  and  rigorous  way  to  specify 
structure.  However,  structure  alone  in  not  enough.  We  need 
to  attach  meaning  to  structure.  The  following  chapters  are 
devoted  to  the  examination  and  development  of  methods  which 
can  take  advantage  of  this  structure. 
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CHAPTER  3  -  THE  DESIGN  APPROACH 


In  this  chapter  we  present  our  approach  to  the  design 
of  programs,  based  on  a  method  developed  by  Michael  Jackson 
[75].  First  we  present  Jackson's  method  and  then  in  the 
appraisal  discuss  how  our  method  differs. 


3.1  The  Jackson  Method 

In  Jackson's  view,  there  are  essentially  two  types  of 
program  operations:  those  directly  concerned  with  the  task 
to  be  performed  and  those  more  concerned  with  the  handling 
of  the  input  and  output  data  structures.  For  example, 
consider  a  program  that  reads  items  from  a  file  and  produces 
invoices.  Of  the  first  type  is  an  operation  such  as  "add 
price  to  invoice  total",  executed  for  each  item  in  the 
invoice.  Operations  of  the  second  type  are:  "read  item 
file",  executed  when  the  next  item  record  is  requested,  and 
"skip  to  new  page",  executed  at  the  start  of  a  new  invoice. 

In  general,  both  types  of  operations  can  be  associated 
with  components  of  the  input  and  output  data  structures. 
The  Jackson  Method  is  based  on  the  premise  that  the  program 
structure  should  mirror  the  structure  of  the  data  it 
processes.  Therefore,  the  objective  is  to  ensure  that 
proper  data  structures  be  developed  so  that  there  will  be  an 
appropriate  program  component  in  which  each  operation  can  be 
placed.  For  example,  "add  price  to  total"  should  appear  in 
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a  component  that  processes  each  item  in  an  invoice  and  "skip 
to  new  page"  in  a  component  that  processes  the  start  of  each 
invoice. 

The  Jackson  method  is  based  on  the  following  three  step 
procedure : 

(1)  Formalize  a  description  of  the  data  to  be  processed. 

(2)  Construct  a  program  structure  based  on  the  data 
structures . 

(3)  Define  the  task  to  be  performed  in  terms  of 

elementary  operations,  list  the  operations,  and 
assign  each  operation  to  a  suitable  component  of  the 
program  structure. 

When  the  program  structure  is  formed  from  the  data 
structures,  it  may  be  incomplete,  as  some  components  may 
need  to  be  added  when  the  executable  operations  are 
assigned. 

In  step  1,  a  formal  structure  must  be  more  than  just  a 
valid  description  of  the  data;  each  formal  structure  must 
express  the  viewed  logical  structure  and  represent  all  of 
the  relationships  relevant  to  the  processing  of  the  data. 
The  expression  grammar  examples  of  the  previous  chapter 
demonstrate  this.  More  than  just  describing  the  set  of 
possible  expressions,  each  grammar  structures  an  expression 
according  to  the  rules  of  operator  precedence. 

Hierarchical  decomposition  is  the  structuring  method 
used  by  Jackson  in  steps  1  and  2.  Components  of  a  structure 
are  of  two  kinds:  elementary  and  composite.  An  elementary 
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Typically,  more  than  one  data  structure  exists  (usually 
at  least  an  input  data  structure  and  an  output  data 
structure).  When  this  is  the  case,  the  data  structures  are 
fit  together  into  a  program  structure  by  identifying  one-to- 
one  correspondences  between  them.  The  program  structure 
must  take  into  account  all  of  the  data  structures  processed. 
If  any  structure  is  ignored,  it  is  possible  that  there  will 
be  operations  associated  with  that  structure  that  cannot  be 
placed  properly  in  the  program  structure. 

3 . 2  Example  of  the  Jackson  Method 

As  an  example  of  the  Jackson  method,  consider  the 
following  problem.  A  factory  warehouse  issues  and  receives 
parts.  For  each  transaction  there  is  a  data  record  with 
three  fields:  the  part  number,  the  transaction  code 

(’issue*  or  ’receipt*),  and  the  quantity.  The  data  records 
are  available  in  sorted  order  by  part  number.  The  task  is 
to  produce  a  written  summary  of  the  net  movement  for  each 
-  part. 

STEP  1 .  The  input  file  is  the  Parts  Movement  File  (PMF) 
and  the  output  file  is  REPORT.  Their  data  structures,  in 
Jackson’s  ’’diagrammatic  logic”,  are: 
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The  names  of  selection  alternatives  are  suffixed  with  wo", 

and  the  name  of  an  iterated  part  is  suffixed  with  "*"  . 

Successive  levels  in  the  diagram  indicate  refinements  of  the 
unpunctuated  immediately  preceding  level.  Thus,  MVTREC*  is 
the  refinement  of  PARTGROUP  and  PARTGROUP*  is  the  refinement 
of  PMF.  Note  that  the  diagram  indicates  context-free  syntax 
only,  and  not  the  context-sensitive  part  of  the 

specification  that  the  part  groups  be  in  sorted  order. 

STEP2.  The  next  task  is  to  find  a  program  structure 
that  reflects  both  data  structures.  First,  correspondences 
between  the  data  structures  are  made,  PMF  corresponds  to 
REPORT,  since  the  program  processes  one  instance  of  PMF  to 
produce  one  instance  of  REPORT.  Similarly,  PARTGROUP 
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corresponds  to  SUMMARYLINE,  since  there  are  the  same  number 


of  each,  and  each  part 
line.  The  resulting 
logic,  is: 


group  is  used  to  produce  one  summary 
program  structure,  in  diagrammatic 
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In  contrast  to  selection  alternatives,  the  names  of  sequence 
parts  are  not  suffixed  with  ,,0,», 
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The  list  of  executable  operations  is: 

open  PMF 
close  PMF 

read  record  from  PMF 
display  summary  line 

PART#  :=  part  number  of  current  input  record 
NET  :=  0 

NET  :=  NET  +  quantity 
NET  : =  NET  -  quantity 
stop 

In  Jackson's  "schematic  logic",  the  resulting  program 


is: 


PPMF  sea 

open  PMF; 

read  record  from  PMF; 

PPMF  BODY  iter  until  eof-PMF 
PP  ARTGROUP~ seg 

PART#  :=  part-number; 
NET  :=  0; 


PPGBODY  iter  until  eof-PMF 

or  PART##part-number 


PMVTREC  seg 

PM VTRECDAT A 
NET  :  = 
PMVTRECDAT A 
NET  :  = 
PMVTRECDAT A 
read  record 
PMVTREC  e&d 
PPGBODY  end 
display  summary  line; 
PP ARTGROUP  end 
PPMFBODY  end 
close  PMF; 


select  code=' issue 
NET  -  quantity; 
og  code= ' receipt  * 
NET  +  quantity; 
end 

from  PMF; 


i 


stop ; 
PPMF  end 


A  drawback  of  Jackson's  notations  is  that,  in  contrast 


to  the  rcfg  case,  only 
directly  applied  to  a 
grammatical  terms,  that 
allowed,  such  as  (al  a2 


one  structuring  mechanism  may  be 
single  composite  component.  In 
means  RE  subexpressions  are  not 
. ..  an)*  or  (a*  |  b*)  .  The  result 


can  be  the  introduction  of  unwanted  levels  in  the  structure 
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hierarchy.  This  problem  does  not  occur  with  an  rcfg-like 
notation. 

3.3  structure  clashes 

A  5^ructure  clash  occurs  when  a  single  program 
structure  cannot  be  found  to  reflect  both  the  input  and 
output  data  structures,  i.  e.,  when  a  one-to-one 
correspondence  of  data  structures  cannot  be  found.  This  is 
usually  the  result  of  paired  iterated  parts  not 
corresponding  properly;  they  must  match  in  order,  number, 
and  boundary.  A  solution  is  to  construct  two  (or  more) 
simple  program  structures.  The  programs  communicate  as 
separate  phases  through  an  interface.  Conceptually,  the 
interface  may  be  viewed  as  an  intermediate  file  of  records, 
whereby  the  first  program  produces  records  to  be  consumed  by 

the  second.  The  structure  of  the  intermediate  file  is 

/ 

chosen  so  that  it  does  not  clash  with  either  the  input  or 
the  output  data  structures.  Typically,  the  two  programs 
will  place  a  different  structural  interpretation  on  the 
intermediate  file.  If  necessary,  additional  intermediate 
files  and  program  phases  are  introduced  to  resolve  any 
further  structure  clashes. 

flcKeeman  terms  this  kind  of  module  structuring 
"feedback-free  modularization",  A  good  discussion  of  his 
point  of  view  can  be  found  in  [  HcKeeman  74], 

The  following  describes  a  typical  structure  clash.  One 
data  structure  views  the  input  data  as  a  seguence  of 
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structured  objects,  while  the  data  really  consists  of  a 
sequence  of  data  cards.  Although  the  atomic  elements  of 
both  objects  and  cards  are  characters,  there  is  no 
particular  relationship  between  objects  and  cards.  An 
object  may  begin  and  end  anywhere  on  a  card  and  may  span 
several  cards;  several  objects  may  share  a  card.  A  solution 
is  to  construct  an  intermediate  file  of  tokens,  in  terms  of 
which  both  cards  and  objects  are  easily  characterized.  We 
usually  choose  a  token  to  be  the  largest  such  unit.  At  the 
very  least,  a  token  can  be  a  single  character. 

This  solution  is  based  on  an  idea  we  call  tokenizing . 
Tokenizing  is  a  technique  for  inter-modular  communication 
whereby  some  possibly  non-atomic  component  is  treated  as  an 
indivisible  basic  unit  (i.e.,  a  token)  in  one  module  and  has 
its  representation  supplied  by  another  module.  Certain 
restrictions  can  be  imposed  to  make  this  communication 
feedback- free • 

Tokenizing  is  good  for  more  than  just  resolving 
structure  clashes.  It  is  a  technique  that  can  be  used  to 
insulate  the  consumer  module  from  representational  details, 
which  are  provided  by  the  producer  module,  thus  layering  a 
view  of  data. 

If  two  views  of  the  data  conflict,  as  in  a  structure 
clash,  each  view  is  broken  up  into  two  (or  more)  layers. 
The  two  views  share  the  bottom  layer(s)  and  meet  at  the 
level  of  tokens.  Thus,  a  token  can  be  viewed  as  a  kind  of 
greatest  common  denominator  [McGowan  76]. 
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3.4  A££I§is&l  of  the  Jackson  flat^o^, 


Jackson's  de^jgfl  a£JB£2i£k  is  guite  attractive  and 
serves  as  the  basis  for  our  own  design  efforts.  There  are, 
however,  problems  with  the  way  the  design  approach  is 
developed  into  a  desfcjn  method.  It  must  be  emphasized  that 
our  criticisms  are  not  meant  to  detract  from  Jackson's 
method.  The  Jackson  Method  has  proven  to  be  an  immensely 
successful,  self-contained  practical  design  method.  In  both 
its  use  and  its  implementation,  it  demands  only  a  modest 
degree  of  sophistication.  It  is  easily  employed  by 
individual  programmers,  even  those  who  have  only  COBOL 
available.  In  this  thesis,  however,  we  are  trying  to 
develop  methods  and  formalisms  that  are  (hopefully)  closer 
to  a  design  ideal.  It  is  in  this  light  that  our  criticisms 
of  Jackson's  method  should  be  viewed. 

The  fundamental  problem  with  the  Jackson  Method  is  that 
design  concerns  are  too  closely  tied  to  purely 
implementation-dependent  concerns.  This  is  seen  in  a  number 
of  ways.  Since  Jackson  uses  an  rcfg-like  formalism  as  a 
structuring  device  only,  and  not  as  a  generative  grammar, 
the  program  must  explicitly  read  and  parse  the  input  data. 
As  a  result,  parsing  concerns  must  be  incorporated  into  the 
program  design.  Since  the  only  parsing  method  considered  is 
based  on  recursive  descent,  backtracking  control  structures 
must  be  introduced  into  the  program  when  a  non-LL  (k) 
situation  is  encountered  so  that  the  data  can  be  handled 
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nondeterministically .  None  of  these  parsing  concerns  are 
inherent  in  the  program  design  and  serve  only  to  cloud  its 
structure . 

Our  approach  in  this  thesis  is  based  on  a  separation  of 
concerns.  Rather  than  just  have  the  program  use  grammar- 
like  structuring  mechanisms,  we  want  it  to  be  an  actual 
grammar.  With  the  grammatical  approach,  the  program 
structures  the  input  and  indicates  how  the  input  is  to  be 
processed;  it  does  not  parse  the  input.  Operations  that 
merely  handle  the  input  data  are  eliminated.  The  structure 
of  the  program  implicitly  specifies  a  parser,  which  is 
supplied  by  a  parser  generator  system.  The  parser  generator 
system  "compiles"  the  program  into  an  executable  version. 
Without  parsing  considerations,  the  program  structure  is 
more  transparent  and  higher-level. 

This  approach  allows  an  elegant  separation  of  design 
concerns  from  implementation  concerns.  The  designer  can 
concentrate  his  efforts  on  his  primary  concern,  the  creative 
process  of  designing  a  proper  program  structure  and  placing 
operations  within  it.  The  secondary  concern,  the 
automatable  process  of  making  the  design  executable,  is 
localized  within  the  parser  generator  system.  Parsing 
considerations  intrude  upon  the  program  design  only  when  the 
available  parser  generator  system  cannot  handle  the  given 
program  (i.e. ,  the  given  grammar). 

As  a  result  of  this  division  of  labor,  each  component 
of  the  complete  system  can  be  specialized  to  perform  its 
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corresponding  task  more  effectively.  From  the  design  side, 
this  means  that  a  formalism  can  be  chosen  that  is  best  able 
to  specify  and  add  meaning  to  structure.  From  the  parsing 
side,  this  opens  the  door  to  a  wide  variety  of  sophisticated 
parsing  algorithms  and  optimizations  that  in  practice  are 
too  complicated  or  tedious  to  be  implemented  by  an 
individual  programmer. 

The  choice  by  Jackson  of  a  recursive  descent-like 
parser  is  no  coincidence:  it  is  an  intuitive,  easily 
implemented  algorithm,  hence  easily  incorporated  into  a 
program.  However,  many  grammars  that  must  be  parsed 
nondeterministically  with  this  method  can  be  parsed 
deterministically  by  other  methods  (such  as  LR) . 

Another  example  of  Jackson's  implementation-dependent 
program  design  is  the  producer-consumer  resolution  of  a 
structure  clash.  There  are  three  basic  ways  to  coordinate 
the  execution  of  a  producer-consumer  pair:  as  disjoint, 
sequentially  scheduled  programs  that  communicate  through  an 
intermediate  file,  as  coroutines  that  share  a  common  buffer, 
or  as  cooperating  parallel  processes  that  share  an  access- 
controlled  common  buffer.  However,  this  decision  need  not 
be  inherent  in  the  design  of  the  programs.  The  design 
should  be  as  insulated  as  possible  from  such  implementation- 
dependent  decisions,  not  structured  around  them,  as  in 
Jackson's  method.  A  formalism  that  cannot  hide 
implementation-dependent  decisions  makes  the  programmer's 
job  harder. 
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Jackson’s  notation  for  semantic  specification  suffers 
serious  drawbacks.  First  and  foremost,  there  is  no  way  to 
parameterize  a  component  part.  The  result  is  that  either  a 
component  is  designed  for  a  specific  instance  and  cannot  be 
reused  in  similar  but  not  identical  situations,  or  it  must 
communicate  through  a  global  variable  interface.  In  either 
case,  component  modularity  is  severely  compromised. 
Furthermore,  there  is  no  "local  scoping"  within  each 
component  part;  all  variables  are  global.  This  forces  the 
programmer  to  build  and  maintain  structures  that  provide  a 
local  scoping  effect.  I  better  formalism  would  provide 
local  scoping  implicitly  and  eliminate  this  unnecessary 
implementation-oriented  distraction.  A  more  detailed 
discussion  of  these  problems  is  given  in  the  next  chapter. 

In  summary,  the  Jackson  Method  often  forces  the 
programmer  to  distort  the  program  structure.  A  proper 
division  of  labor,  with  a  concomitant  specialization,  helps 
eliminate  nonessential  concerns,  while  at  the  same  time 
promoting  independent  modularity.  The  grammatical  approach 
encourages  a  program  design  that  is  easier  to  write,  easier 
to  understand,  and  more  likely  to  be  correct. 

In  this  chapter  we  have  detailed  our  design  approach 
and  discussed  some  of  the  characteristics  a  suitable 
formalism  must  possess.  The  grammatical  formalisms  of  the 
previous  chapter  are  inadequate  to  meet  our  needs,  but  as  we 
show  in  the  next  chapter,  they  can  be  enhanced. 
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CHAPTER  4  -  SOflE  ENHANCED  GRAQJJARS 


For  our  purposes,  context-free  grammars  are  deficient 
in  two  respects: 

(1)  As  the  name  "context-free”  implies,  cfg's  provide  no 
way  to  specify  context-sensitive  languages. 

(2)  Cfg's  provide  no  way  to  link  meaning,  or  semantics, 
to  syntax. 

Often,  a  language  can  be  described  by  a  context-free 
phrase  structure  with  certain  context-sensitive  constraints. 
In  the  compiler  literature,  context-sensitive  constraints 
are  often  referred  to  as  "static  semantics".  This  term  seems 
misleading,  and  we  refrain  from  using  it.  As  an  example  of 
such  a  language,  consider  a  nonempty  sequence  of  record 
groups.  Each  group  consists  of  a  nonempty  sequence  of 
records  that  possess  the  same  key,  and  two  consecutive 
groups  contain  records  that  possess  different  keys.  Such  a 
language  cannot  be  described  by  a  context-free  grammar, 
though  a  cfg  can  be  used  to  describe  the  language's  phrase 
structure  (i.e.  a  sequence  consists  of  record  groups,  and  a 
group  consists  of  records)  .  Neither  can  a  cfg  be  used  to 
describe  the  processing  associated  with  a  sequence. 

On  the  other  hand,  cfg's  have  many  advantages,  some  of 
which  were  discussed  in  Chapter  2.  Context-free  grammars 
are  both  an  easy-to-use,  easy- to-under stand  tool  and  a 
rigorous  mathematical  object.  The  ubiquitous  use  of  cfg's 
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in  sources  from  programming  manuals  to  formal  language 
theory  textbooks  attests  to  their  usefulness. 

What  we  want  is  a  formalism  based  on  the  context-free 
grammar  that  addresses  these  deficiencies.  In  this  chapter 
we  present  some  of  the  alternatives  proposed,  including 
translation  grammars,  attributed  translation  grammars,  and 
affix  grammars. 

When  a  grammar  is  used  to  process  an  input,  we  assume 
that  the  grammar  attempts  to  parse  the  entire  input.  In 
grammatical  terms,  this  means  that  an  implicit  endmarker 
symbol  is  placed  at  the  end  of  the  input  and  at  the  extreme 
right  end  of  the  grammar  rule  whose  left  side  is  the  start 
symbol . 


4 . 1  Translation  Grammars 

The  first  method  examined  is  a  standard  way 
compiler  writers  to  attach  semantics  to  cf-syntax. 
method,  so-called  "action  symbols",  which  represent 
routines,  are  embedded  in  rcfg  rules.  We 
terminology  of  Lewis,  Rosenkrantz  and  Stearns  [Lew 
74]  for  the  grammatical  aspects  of  the  method. 


used  by 
In  this 
semantic 
use  the 
is  et  al. 


4.1.1  Definition  of  Translation  Grammars 

Formally,  a  translation  grammar  G=(Vn,Va  0  Vi,R,S)  is  an 
rcfg  in  which  the  set  of  terminals  is  partitioned  into  a  set 
of  action  or  output  symbols  Va  and  a  set  of  input  symbols 
Vi.  The  strings  in  the  language  generated  by  a  translation 
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grammar  are  called  activity  sequences.  The  grammar 
Gi= (Vn,Vi,Ri, S)  obtained  by  deleting  all  action  symbols  from 
rules  in  R  is  called  the  input  grammar  or  underlying  j:c£q 
for  G . 

Given  an  activity  sequence  a  of  input  and  action 
symbols,  we  refer  to  the  sequence  of  input  symbols  obtained 
by  deleting  all  action  symbols  as  the  input  part,  and  denote 
it  by  IP (a) .  Similarly,  the  action  part  is  the  sequence  of 
action  symbols  obtained  by  deleting  all  input  symbols,  and 
is  denoted  by  AP(a). 

For  each  activity  sequence,  the  action  part  is  called 
the  translation  of  the  input  part.  Each  activity  sequence 
pairs  an  input  part  with  an  output  part.  The  set  of  all 
such  pairs  obtained  from  a  translation  grammar  G  is  called 
the  syntax -directed  translation  T (G)  defined  by  G. 
Formally, 

T  (G)  =  {  ( b, c)  |  a€L  (G)  ,  b=IP  (a)  ,  and  c=AP(a)} 

The  set  of  syntax-directed  translations  defined  by 
translation  grammars  is  the  same  set  defined  by  the  simple 
syntax-directed  transductions  of  Lewis  &  Stearns  [68]  and 
the  simple  syntax-directed  translation  schemata  of  Aho  & 
Ullman  [73],  These  two  formalisms,  however,  do  not  describe 
activity  sequences, 

4.1.2  Activity  Sequences 

An  activity  sequence  does  more  than  just  specify  a 
translation  from  input  part  to  output  part.  It  can  be 


-32- 


viewed  as  an  external  description  of  the  dynamic  behavior  of 
a  language  processor.  The  occurrence  of  an  input  symbol  in 
an  activity  sequence  can  be  interpreted  as  the  consumption 
of  that  symbol  by  the  processor.  (Consumption  is 
distinguished  from  lookahead.)  The  occurrence  of  an  action 
symbol  in  an  activity  sequence  can  be  interpreted  as  the 
emission  of  that  symbol  by  the  processor.  A lternatively , 
the  occurrence  of  an  action  symbol  in  an  activity  sequence 
can  be  interpreted  as  the  calling  of  the  "semantic  action 
routine"  named  by  the  action  symbol,  which  is  the  viewpoint 
we  adopt. 

An  activity  sequence  can  be  thus  viewed  as  specifying 
not  only  the  sequence  of  semantic  routine  calls  (action 
symbol  emissions)  corresponding  to  an  input  part,  but  also 
their  timing  with  respect  to  the  consumption  of  input 
symbols.  It  is  important  to  note  that  this  timing  is 
independent  of  the  method  used  to  parse  the  input  sequence. 

4.1.3  Examples  of  Translation  Grammars 

In  the  first  example  we  give  a  translation  grammar 
G=(Vn,Va  0  Vi, R, Exp)  that  defines  an  inf ix-to-postf ix 
translation  for  expressions  with  operators  «■  and  *.  The 
elements  of  G  are  as  follows. 

Vn  =  [Exp, Term,  Factor} 

Va  =  {Out plus, Outtimes,Outa} 

Vi  =  {(,)  ,  +  ,*,a} 

R  consists  of  the  following  rules: 
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Exp:  Term  (  1  ♦*  Term  Outplus  )* 

Term:  Factor  (  **'  Factor  Outtimes  )♦ 

Factor:  • (*  Exp  ')'  1  'a*  Outa 

Outplus,  Outtimes,  and  Outa  output  '♦•,  •**,  and  ’a', 

respectively,  and  produce  the  inf ix-to-postf ix  translation. 

The  situation  gets  more  complicated  when  we  add  the 
operators  -  and  /  to  this  grammar.  Operators  must  be 
remembered  and  retrieved,  requiring  a  stack.  The  routines 
are  now: 

(a)  Pushplus,  Pushminus,  Pushtimes,  and  Pushdiv,  which 
save  their  respective  operators  on  the  stack 

(b)  Outop,  which  prints  the  operator  on  the  top  of  the 
stack 

(c)  Outa,  which  prints  "a" 

(d)  Pop,  which  pops  the  stack 
The  rules  are  now: 

Exp:  Term  (  (•♦•  Pushplus  |  ■-•  Pushminus) 

Term  Outop  Pop  ) * 

Term:  Factor  (  (»*•  Pushtimes  |  •/'  Pushdiv) 

Factor  Outop  Pop  ) * 

Factor:  •  ('  Exp  •  )•  1  *a*  Outa 

Thus,  for  example,  associated  with  the  input  part 

a  -  a  *  a 

is  the  activity  sequence 

a  Outa  -  Pushminus  a  Outa  *  Pushtimes  a  Outa  Outop 
Pop  Outop  Pop 

and  the  action  part: 

Outa  Pushminus  Outa  Pushtimes  Outa  Outop  Pop  Outop  Pop. 
When  executed,  the  action  part  will  output  "a  a  a  *  -",  the 
Polish  translation  of  the  input  part. 
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It  should  be  noted  that  it  is  possible  to  give  a 
different  set  of  rules,  based  on  a  different  expression 
input  grammar,  for  which  the  stack  is  not  necessary.  For 
the  present  grammar,  however,  the  stack  is  required. 

As  another  example,  consider  the  record-group  language 
described  above.  The  context-sensitive  constraints  of  this 
language  cannot  be  described  by  a  translation  grammar. 
However,  we  can  use  a  translation  grammar  to  describe  the 
phrase-structure  and  to  impose  a  general  semantic  structure 
as  follows: 

Data:  Initdata  (Group) ♦  Processdata 

Group:  Initgroup  (Rec) +  Processgroup 

Rec:  Initrecord  Record  Processrecord 

The  action  symbols  are  those  symbols  prefixed  by  Init  or 
Process. 

4.1.4  Parsing  Translation  Grammars 

To  determine  whether  a  translation  grammar  can  be 
handled  by  an  LL  or  LR  parser,  we  simply  treat  each  action 
symbol  as  if  it  defined  a  null  nonterminal.  That  is,  we  can 
transform  the  translation  grammar  G=(Vn,Va  0  Vi,R,S)  into  an 
ref g  G»=(Vn  0  Va,Vi,R»,S)  where  R'  =  R  U  {X:e  |  X€Va) .  The 
translation  grammar  G  is  defined  to  be  LL  (k)  (LR(k))  if  and 
only  if  G'  is  LL  (k)  (LR  (k)  )  •  The  action  associated  with 
action  symbol  X  is  performed  upon  recognition  of  the  grammar 
rule  X:  e. 

The  following  two  theorems  provide  some  guidelines 
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about  whether  a  given  translation  grammar  is  LL  (k)  or  LR(k). 
The  first  says  that  a  translation  grammar  G  is  LL  (k)  if  and 
only  if  its  input  grammar  Gi  is  LL  (k)  .  Since  g-rules  do  not 
affect  FIRST_k  sets,  it  follows  that  e-rules  can  be  added  or 
deleted  without  affecting  LL(k)-ness  (or  non-LL  (k) -ness) .  If 
the  grammar  was  LL  (k )  before  insertion  or  deletion  of  e- 
rules,  it  still  is  after.  Likewise,  if  it  was  not  LL (k) 
before,  it  is  not  after.  This  theorem  means  that  an  LL (k) 
parser  for  the  translation  grammar  G  need  not  actually 
construct  the  derivation  X=>e  as  dictated  by  G*;  it  can 
simply  perform  the  action  associated  with  X  as  soon  as  it 
sees  X,  and  then  proceed  to  the  next  symbol. 

The  situation  is  not  as  simple  for  the  LR  case.  The 
theorem  here  goes  only  one  direction:  a  translation  grammar 
G  is  LR (k)  only  if  the  input  grammar  Gi  is  L R ( k)  .  To  prove 
this  it  is  necessary  to  show  that  LR  (k)  properties  are 
preserved  when  e-rules  are  removed.  The  proof  relies  on  a 
theorem  in  [Aho  5  Oilman  72]  that  phrases  the  LR  (k) 
condition  in  terms  of  such  LR  notions  as  valid  items,  viable 
prefixes,  and  e-FREE-FIRST_k  sets.  While  interesting,  the 
proof  is  not  particularly  relevant  here,  and  is  omitted. 

We  next  show  by  counter-example  that  the  converse  of 
the  LR  theorem  is  not  true;  that  is,  the  insertion  of  e- 
rules  may  cause  a  previously  LR  (k)  grammar  to  become  non- 
LR(k).  Consider  the  translation  grammar 
Gn=({S},{X)  0  (a,b},R,S)  where  X  is  an  action  symbol  and  R 
consists  of: 
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S:  (a  power  n)  a  |  X  (a  power  n)  b 
The  associated  rcfg  for  Gn  is 

Gn*=({S}  U  (X},{a,b},R  0  {X:e},S).  Despite  the  fact  that 
the  input  grammar  Gni  is  LR(0)  for  all  n,  Gn  •  is  not  LR  (k) 
for  any  k  <  n.  Intuitively,  this  is  so  because,  for  k<n, 
examination  of  (a  power  k)  cannot  resolve  whether  X  should 
be  recognized  from  e  or  not. 

This  example  shows  that,  in  contrast  to  the  cfg  case, 
the  classes  of  activity  sequences  describable  by  LR (k) 
translation  grammars  are  distinct  for  every  k.  For 
translation  grammars,  k+1  is  indeed  better  than  k. 

4.1.5  Appraisal  of  Translation  Grammars 

While  a  start  in  the  right  direction,  translation 
grammars  do  not  meet  our  objectives.  First,  the  problem  of 
context-sensitive  constraints  is  ignored  altogether. 
Although  it  is  possible  for  a  semantic  routine  to  perform  an 
action  based  on  some  context-sensitive  condition,  this  is 
not  the  same  as  imposing  context-sensitive  constraints  on 
the  grammar  rules. 

Second,  the  method  of  specifying  actions  is 
unsatisfactory:  there  is  no  way  to  specify  the  flow  of 

semantic  data.  All  such  data  must  be  stored  in  global 
variables  whose  names  are  not  visible  in  the  grammar,  but 
which  must  be  known  to  all  routines  that  use  them.  Since 
the  semantic  routines  must  communicate  through  global 
variables  rather  than  parameters,  modules  are  tightly 
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coupled.  Furthermore,  the  method  presupposes  some 
environment  in  which  the  actions  are  performed  and  in  which 
the  global  variables  reside. 

Likewise,  the  method  itself  provides  no  way  for  the 
semantic  routines  to  access  data  produced  by  preceding 
modules  (such  as  the  lexical  scanner)  or  to  pass  data  to  be 
consumed  by  later  modules.  Instead,  global  variable 
interfaces  must  be  constructed  by  the  designer.  This 
problem  is  even  more  serious  since  it  involves  modules  not 
under  the  translation  grammar  user's  control. 

For  our  purposes,  the  fundamental  weakness  of 
translation  grammars  is  the  global  nature  of  semantic 
information.  The  remedy  involves  parameterization  and 
"local  scoping"  within  each  grammar  rule.  Parameterization 
provides  visible,  controlled  interfaces  for  inter-component 
communication.  Without  parameterization,  the  designer  must, 
in  general,  build,  maintain,  and  enforce  his  own  global 
variable  interfaces.  If  a  given  semantic  routine  is  such 
that  only  constants  are  passed  as  inputs  (i.e.  no 
variables) ,  then  the  single  routine  can  be  replaced  by  a  set 
of  special  purpose  routines,  one  for  each  possible  input 
tuple.  This  alternative  eliminates  global  input  variables 
at  the  expense  of  a  polynomial  increase  in  the  number  of 
routines . 

Local  scoping  provides  each  instance  of  a  component 
with  local  information  not  accessible  to  components  within 
which  the  given  component  is  contained.  Without  local 
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scoping,  this  local  information  must  be  maintained 
"manually”.  However,  if  the  formalism  allows  local 
information  to  be  associated  with  component  instances,  the 
local  information  is  managed  "automatically”  by  the 
structure  of  the  derivation  tree. 

The  translation  grammar  examples  illustrate  some  of 
these  points.  In  the  expression  grammar  with  + ,  -,  *,  and  / 
as  operators,  separate  routines  for  each  of  the  four 
operators  were  written  because  we  could  not  have  a  single 
"Push"  routine  parameterized  by  the  current  input  token. 
Furthermore,  the  explicit  stack  would  have  been  eliminated 
had  there  been  local  scoping  within  each  rule.  The 
structuring  desired  is  provided  implicitly  by  the  derivation 
tree . 

A  result  of  the  global  nature  of  information  is  that  it 
causes  the  grammar  to  be  cluttered  with  action  symbols  that 
only  serve  to  localize  the  scope  of  information  (i.e., 
provide  the  effect  of  parameterization  or  local  scoping)  , 
but  otherwise  convey  no  semantic  information.  These 
"unnecessary"  action  symbols  obscure  the  designers  intent. 
The  tedious,  error-prone  task  of  supporting  scope 
localization  can  and  should  be  provided  implicitly  by  the 
formalism. 

Parameterization  and  local  scoping,  however,  do  more 
than  simply  eliminate  tedious  tasks.  They  contribute 
greatly  to  the  required,  though  intangible,  objectives  of 
conceptual  clarity,  modularity,  and  component  generality. 
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The  case  against  global  variables  is  well-known  (cf.  [Wulf  S 
Shaw  73]);  parameterization  and  local  scoping  are  an 


effective  alternative. 

What  we  want  is  a  formalism  where  the  ability  to 
specify  semantics  is  bui^t-in  rather  than  tacked- on . 
Translation  grammars  are  fine  for  simple  string-to-str ing 


translations,  but  are  not  adequate  for 

more  sophisticated 

semantic  processing.  Such  sophisticated 

processing  requires 

a  more  sophisticated  formalism. 


Our  effort,  however,  has  not  been 

wasted.  Translation 

grammars  are  a  starting  point,  and  the 

parsing  results  are 

applicable  to  the  more  powerful  grammars  considered  next. 


-40- 


4 . 2  Attributed  Translation  Grammars 

In  the  development  of  a  formalism  to  specify 
sophisticated  syntax-directed  translations,  Lewis, 
Rosenkrantz,  and  Stearns  [74]  generalize  context-free 
grammars  in  two  steps.  We  looked  at  the  first  step, 
translation  grammars,  in  the  previous  section.  In  this 
section  we  look  at  the  second  step,  it tribute d  translation 
grammars  (atg's) ,  which  combine  translation  grammars  and  the 
attribute  gra mmars  of  Knuth  [68].  Attribute  grammars  were 
originally  introduced  as  a  means  for  specifying  the 
semantics  of  context-free  grammars. 

The  basic  idea  behind  atg's  is  that  a  fixed  number  of 
attributes  are  associated  with  each  basic  symbol  of  a 
context-free  translation  grammar.  (For  reasons  to  be 
discussed  later,  we  use  only  cf-like,  and  not  rcf-like, 
rules.)  Attribute  values  are  used  to  convey  ’'semantic” 
information  about  a  particular  basic  symbol.  An  attributed 
symbol  consists  of  a  basic  symbol  together  with  an 
associated  attribute  value  for  each  attribute. 
Notationally ,  the  attribute  values  follow  the  basic  symbol 
in  parentheses.  As  a  simple  example,  suppose  that  CONSTANT 
is  a  basic  symbol  possessing  one  integer- valued  attribute. 
Then  the  particular  attributed  symbol  consisting  of  CONSTANT 
with  attribute  value  45  is  written  as  CONSTANT ( 45) .  In  this 
case,  the  attribute  value  may  be  used  to  specify  the 
numerical  value  of  the  constant  represented  by  an  instance 
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of  the  basic  symbol  CONSTANT.  Attributes  are  given  value 
through  evaluation  rules  associated  with  each  grammar  rule. 

The  attributes  of  nonterminal  and  action  symbols  are 
partitioned  into  two  sets:  inherited  attributes  and 

synthesized  attributes.  The  inherited  attributes  of  a  given 
symbol  X  are  used  to  convey  information  about  X*s  context  in 
a  derivation  tree,  and  the  synthesized  attributes  of  X 
convey  information  about  phrases  derived  from  X  in  the  given 
context. 

4.2.1  Definition  of  Attributed  Translation  Grammars 

An  attributed  translation  grammar  is  a  context-free 
translation  grammar  for  which  additional  specifications  are 
made: 

(1)  Each  input,  nonterminal,  and  action  symbol  has  an 
associated  finite  set  of  attributes,  and  each 
attribute  has  a  (possibly  infinite)  domain. 

(2)  Each  nonterminal  and  action  symbol  attribute  is 
classified  as  being  either  inherited  or  synthesized. 

(3)  The  evaluation  of  the  attributes  is  defined  within 
the  scope  of  a  single  grammar  rule. 

(a)  For  each  occurrence  of  an  inherited  attribute  on 
the  right  side  of  a  given  grammar  rule,  there  is 
an  associated  evaluation  rule  that  says  'how  to 
compute  a  value  for  that  attribute  as  a  function 
of  certain  other  attributes  occurring  in  the 
given  grammar  rule. 
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(b)  Similarly,  for  each  occurrence  of  a  synthesized 
attribute  on  the  left  side  of  a  given  grammar 
rule,  there  is  an  associated  evaluation  rule 
that  says  hov  to  compute  a  value  for  that 
attribute  as  a  function  of  certain  other 
attributes  occurring  in  the  given  grammar  rule. 

(c)  For  each  synthesized  action  symbol  attribute, 
there  is  an  associated  rule  that  says  hov  to 
compute  a  value  for  that  attribute  as  a  function 
of  certain  other  attributes  of  the  action 
symbol . 

(d)  An  initial  value  is  specified  for  each  inherited 
attribute  of  the  start  symbol. 

An  attributed  translation  grammar  defines  an  attributed 
derivation  tree  in  the  following  manner. 

(1)  The  underlying  translation  grammar  is  used  to 
construct  an  unattributed  derivation  tree  for  an 
unattributed  activity  sequence. 

(2)  Attribute  values  are  assigned  to  the  occurrences  of 
input  symbols  in  the  tree. 

(3)  The  inherited  attributes  of  the  start  symbol  are 
initialized . 

(4)  The  attribute  evaluation  rules  are  employed  wherever 
possible  in  an  attempt  to  fill  in  the  attribute 
values  for  all  nonterminal  and  action  symbols  in  the 
tree. 

Atg's  are  a  very  general  formalism.  They  are  so 
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general,  in  fact,  that  it  is  possible  for  an  atg  to  contain 
circularities,  in  which  case  the  last  step  above  cannot 
succeed.  An  atg  is  said  to  be  well-defined  if  and  only  if 
for  any  derivation  tree  obtained  from  the  underlying 
translation  grammar,  the  process  described  above  can  be  used 
to  compute  a  value  for  each  attribute  of  each  symbol  in  the 
tree.  The  test  for  well-definedness  of  [ Knuth  68]  can  be 
extended  straightforwardly  to  atg's.  In  practice,  we  are 
interested  only  in  well-defined  atg's,  and,  as  such, 
restrict  our  attention  to  certain  sub-classes  that  are 
guaranteed  to  be  well-defined. 

4.2.2  Examples  of  Attributed  Translation  Grammars 

We  first  demonstrate  the  semantic  superiority  of  atg's 
over  translation  grammars  with  a  trivial  example,  an  atg  for 
the  inf ix-t o- postf i x  translation  of  expressions  with 
operators  ♦,  -,  *,  and  /. 

Since  regularized  rules  are  not  allowed,  the  expression 
-syntax  is  formulated  in  the  LL  manner  of  Chapter  2.  All 
attributes  are  character- valued .  The  nonterminals  Addop  and 
Mulop  each  have  a  synthesized  attribute,  and  the  action 
symbol  Out  has  an  inherited  one.  A  semicolon  is  used  to 
separate  a  grammar  rule  from  its  associated  evaluation  rule. 
The  rules  are  as  follows: 

Exp:  Term  Termlist 

Termlist:  Addop(opl)  Term  Termlist  0ut(op2); 
op2  :=  op  1 

Termlist:  e 
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Term:  Factor  Factorlist 


Factorlis t:  Mulop  (opl)  Factor  Factorlist  Dut(op2); 

op2  :=  opl 

Factorlist:  e 

Factor:  1  ('  Exp  • )  * 

Factor:  Constant ( val 1 )  0ut(val2); 
val2  :=  vail 


Addop  (op)  : 

; 

op  :  = 

•  + » 

Addop  (op)  : 

i  _ »  . 

• 

op  :  = 

i  _ » 

Mulop  (op)  : 

i  *»  • 

* 

op  :  = 

» * » 

Mulop  (op)  : 

V' ; 
op  :  = 

V 

With  atg’s,  attributes  are  actually  part  of  the 
derivation  tree,  so  we  are  now  able  to  take  full  advantage 
of  the  syntactic  structure.  Since  evaluation  rules  are 
defined  within  the  scope  of  a  single  grammar  rule,  the  stack 
of  the  earlier  translation  grammar  version  is  not  needed. 
Action  symbol  parameterization  means  only  one  Out  symbol  is 
needed.  Furthermore,  we  are  no  longer  limited  to  the  single 
operand  "a"  -  any  attributed  constant  can  be  used.  The 
clumsiness  of  the  notation  yill  be  addressed  later. 

If  the  record-group  example  were  rewritten  with  an  atg, 
each  symbol  would  be  parameterized  by  the  attributes  with 
which  it  deals  and  only  those  attributes.  There  are  no  more 
"mysterious"  global  variables.  Thus,  atg's  meet  the 
parameterized  interface  reguirements.  However,  as 
presented,  there  is  still  no  way  to  specify  context- 
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sensitive  constraints,  e.g.  to  ensure  that  records  in 
group  have  the  same  key  and  that  consecutive  groups  have 
differing  keys. 

4.2.3  Performing  Attributed  Translations 

The  problem  of  finding  an  order  for  performing  the 
attribute  evaluation  rules  can  be  very  complicated.  If  the 
grammar  contains  circularities,  then  such  an  order  does  not 
exist  and  the  grammar  is  not  well-defined. 

Since  the  decision  algorithm  for  well-definedness  runs 
in  exponential  time  as  a  function  of  the  size  of  the 
grammar,  we  seek  easily  tested  sufficient  conditions  for 
well  definedness.  The  conditions  developed  by  Bochmann  [76] 
cover  a  large  class  of  atg's.  For  grammars  in  this  class, 
all  attributes  can  be  evaluated  in  a  fixed  number  of  left  to 
right  passes  over  an  (initially  unattributed)  derivation 
tree.  The  number  of  passes  required  is  easily  computed  from 
the  grammar. 

The  conditions  developed  independently  by  Lewis  et  al. 
[74]  coincide  with  Bochmann*s  conditions  for  attribute 
evaluation  in  a  single  left  to  right  pass  over  the  syntax 
tree.  A  grammar  that  satisfies  these  conditions,  in  Lewis 
et  al.'s  terminology,  is  called  an  L-a t tribute d  translation 
grammar  (L-atg)  . 


Together 

,  the  L-atg 

conditions  mean 

that  for 

a 

given 

rule  A:  XI  X2 

...  Xn,  the 

attributes  can 

be  evaluated 

in  a 

single  left 

to  right  pass  over  the  derivation 

tree  as 
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follows 


evaluate  the  inherited  attributes  of  A 
for  i  =  1  to  n  do 

evaluate  the  inherited  attributes  of  Xi 
evaluate  the  synthesized  attributes  of  Xi 
end  do 

evaluate  the  synthesized  attributes  of  A 
Formally,  the  L-atg  conditions  are: 

(1)  For  each  attribute  evaluation  rule  for  an  inherited 
attribute  of  a  given  symbol  in  the  right  side  of  a 
grammar  rule,  each  argument  of  that  evaluation  rule 
is  either  an  inherited  attribute  of  the  nonterminal 
on  the  left  side  or  an  arbitrary  attribute  of  a 
right  side  symbol  that  appears  to  the  left  of  the 
given  symbol. 

(2)  For  each  attribute  evaluation  rule  for  a  synthesized 
attribute  of  a  given  left  side  nonterminal,  each 
argument  of  that  evaluation  rule  is  either  an 
inherited  attribute  of  the  given  nonterminal  or  an 
arbitrary  attribute  of  a  right  side  symbol. 

(3)  For  each  attribute  evaluation  rule  associated  with  a 
synthesized  attribute  of  a  given  action  symbol,  each 
argument  of  that  rule  is  an  inherited  attribute  of 
that  action  symbol. 

(The  "L"  in  the  term  "L-attributed"  comes  from  the  "left” 
restriction  in  condition  1.) 

Conditions  2  and  3  ensure  that  the  grammar  is  well- 
defined.  Condition  1  ensures  that  the  inherited  attributes 
of  a  given  node  in  the  derivation  tree  depend  (either 
directly  or  indirectly)  only  on  input  symbol  attributes  that 
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occur  to  the  left  of  the  given  node.  It  also  ensures  that 
the  synthesized  attributes  of  a  given  node  depend  only  on 
input  symbol  attributes  that  occur  to  the  left  or  below  the 
given  node.  In  essence,  condition  1  means  that  inherited 
attributes  carry  left  context  and  synthesized  attributes 
carry  information  about  a  phrase  derived  by  the  given  symbol 
in  the  given  left  context. 

Lewis  et  al.  define  two  sub-classes  of  L-atg*s  for 
which  syntax  analysis  and  attribute  evaluation  can  be  done 
together  deterministically  in  a  single  pass  from  left  to 
right.  The  first  class  requires  the  underlying  translation 
grammar  to  be  LL  ( k) .  (This  is  true  if  and  only  if  the  input 
grammar  is  LL(k).)  In  the  second  class,  the  underlying 
translation  grammar  must  be  postfix  LR ( k)  and  all  attributes 
must  be  synthesized.  A  postfix  translation  grammar  is  one 
in  which  all  action  symbols  occur  only  at  the  extreme  right 
end  of  grammar  rule  right  sides.  (Because  of  its  special 
form,  a  postfix  translation  grammar  is  L R  ( k)  if  and  only  if 
its  input  grammar  is  LR  (k)  . )  LL  (k)  and  LR  (k)  parsers  can  be 
extended  straightforwardly  to  handle  these  sub-classes  of  L- 
atg's,  respectively.  However,  in  Section  4.3.4  we  show  that 
a  much  wider  class  of  L-atg's  can  be  handled 
deterministically  in  a  single  left  to  right  combined  syntax 
analysis  and  attribute  evaluation  pass. 

4.2.4  Appraisal  of  Attributed  Translation  Grammars 

Attributed  translation  grammars  have  been  used  as  a 
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method  for  defining  the  semantics  of  programming  languages. 
In  this  respect,  they  have  been  criticized  as  being  too 
"operational”.  Our  purpose,  however,  is  to  study  the  use  of 
grammatical  formalisms  as  programming  languages  and  to  apply 
these  formalisms  to  programming.  From  this  point  of  view, 
their  algorithmic  nature  is  an  advantage. 

The  left  to  right  orientation  of  the  L-attributed 
restriction  is  a  natural  one  from  the  programming  language 
viewpoint.  The  effect  of  a  multi-pass  translation  can  be 
achieved  by  cascading  a  sequence  of  L-atg's,  whereby  the  ith 
grammar  is  the  producer  of  data  for  the  i+lth  and  the 
consumer  of  data  from  the  i-lth.  Such  producer-consumer 
relationships  are  a  familiar  notion.  This  same  approach  was 
used  in  Section  3.3  to  resolve  structure  clashes. 

Atg's,  as  we  have  seen,  provide  clean,  parameterized 
interfaces.  The  notions  of  inherited  and  synthesized 
attributes  correspond  closely  to  the  notions  of  input  and 
output  parameters.  This  is  especially  true  for  grammars 
that  meet  condition  3  of  the  L-atg  definition. 

Since  attributes  are  actually  part  of  the  derivation 
tree,  the  semantic  routines  are  now  able  to  take  advantage 
of  the  syntactic  structure.  As  a  result,  semantics  are  not 
just  tacked  onto  the  syntax  description,  but  rather  are 
incorporated  as  an  integral  part. 

What  of  the  negative  points?  First,  the  notation  is 
rather  clumsy.  .  Since  each  instance  of  an  attribute  variable 
in  a  grammar  rule  must  be  unique,  many  evaluation  rules  turn 
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out  to  be  simple  copies.  We  would  prefer  a  notation  that 
eliminates  the  clutter  of  simple  copy  rules  by  making  them 
implicit. 

As  presented,  atg's  cannot  specify  context-sensitive 
constraints.  The  typical  solution  is  to  give  for  each 
grammar  rule  a  set  of  semantic  conditions  in  addition  to  the 
evaluation  rules.  Each  condition  is  a  relation,  or 
predicate,  on  the  attributes  in  the  grammar  rule  that  must 
be  satisfied  in  each  application  of  the  grammar  rule. 
However,  the  solution  fosters  a  proliferation  of  rules  and 
makes  the  notation  even  more  cluttered.  We  would  prefer  a 
more  unified  notation. 

Regularized  rules  cannot  be  used  because  of  renaming 
problems  introduced  by  the  *  operator.  For  example,  there  is 
no  way  to  distinguish  among  the  multiple  copies  of  the  same 
attributed  symbol  in  a  specific  instance  of  a  *-rule  in  a 
derivation  tree.  The  renaming  problems  are  solved 
implicitly  by  a  recursive  formulation  of  * .  After  we 
enforce  a  more  imperative  view  of  grammars  in  Chapter  5, 
such  renaming  problems  disappear  and  regularized  rules  will 
be  used. 

Another  drawback  of  atg*s  is  the  heavy  reliance  on 
other  notations,  specifically  the  use  of  a  conventional 
programming  language  for  specifying  evaluation  rules.  In 
Chapter  5  we  develop  a  notation  which  integrates  syntax  and 
semantics  more  completely.  As  will  be  seen,  all  the 
necessary  tools  are  already  present. 
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4 . 3  Affix  Grammars 

The  affix  grammars  (ag's)  of  Roster  [71]  are  closely 
related  and  formally  equivalent  to  the  attributed 
translation  grammars  just  presented.  The  more  unified 
approach  of  affix  grammars  overcomes  many  of  the  objections 
to  atg's  and  deserves  close  attention. 

Ag’s  extend  cfg's  to  allow  explicit  control  of  semantic 
and  syntactic  data  by  parameterization.  Each  nonterminal  in 
an  ag  may  have  associated  with  it  a  fixed  number  of 
parameters  called  affix -positions,  which  are  attributes  of 
the  corresponding  language  construct.  In  the  rules  of  an 
ag,  an  affix-position  may  be  occupied  by  either  a  specific 
value  called  an  affix,  or  an  affix- variable  that  stands  in 
place  of  an  affix.  Affixes  and  affix  variables  may  be 
passed  as  parameters  to  primitive  £redicates,  which  provide 
the  means  of  imposing  context-sensitive  constraints  and 
specifying  semantics.  A  rule  of  systematic  substitution  of 
affixes  for  affix- variables  in  the  ag  rules  is  used  to 
generate  cf-like  grammar  rules,  which  in  turn  determine  the 
language  generated  by  the  grammar.  (Such  a  scheme  is  called 
a  two-level  grammar.) 

The  distinction  between  inherited  and  derived  affixes 
in  ag's  corresponds  directly  to  the  distinction  between 
inherited  and  synthesized  attributes  in  atg's.  An  inherited 
affix,  written  preceded  by  a  down-arrow  "f",  carries 
information  about  context;  a  derived  affix,  written  preceded 
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by  an  up-arrow  "T",  carries  information  about  a  derived 
phrase  in  the  given  context. 

4.3.1  Affix  Grammar  Examples 

Since  the  formal  definition  of  an  ag  is  rather  hard  to 
understand,  we  develop  a  few  examples  to  provide  an  informal 
feel  for  ag's.  The  first  example  is  based  on  one  from  Watt 
[  74,77a]. 

Suppose  we  wish  to  define  a  skeleton  language  L  in 
which  "tags"  (e.g.  names  or  identifiers)  may  be  "defined" 
and  "used"  under  the  following  restrictions:  all  tags  must 
be  defined  before  use  and  no  tag  may  be  defined  more  than 
once.  For  example  we  wish  to  allow  sentences  such  as  "begin 
define  x  use  x  end"  but  not  "begin  define  x  use  y  end"  or 
" begin  define  x  define  x  use  x  end".  For  the  purposes  of 
this  example,  we  will  be  concerned  only  with  the  problem  of 
specifying  such  a  language  and  not  with  the  semantic 
processing  of  tags.  In  addition,  we  impose  the  further 
•restriction  that  all  tags  be  defined  before  any  are  used. 
After  reading  the  example,  the  reader  should  have  no  trouble 
developing  an  ag  that  does  not  impose  this  additional 
restriction. 

The  language  generated  by  the  following  cfg  ignores  the 
context-sensitive  constraints  of  L  as  it  provides  an 
envelope  for  L,  i.e.  L  is  a  proper  subset  of  L(G) . 

( r  1 )  data:  'begin'  dlist  ulist  'end' 

(r2)  dlist:  1  define1  tag 
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(r3)  dlist:  dlist  'define*  tag 
(r4)  ulist:  e 

(r5)  ulist:  ulist  'use'  tag 
(r6)  tag:  'x* 

(r7)  tag:  'y' 

(r8)  tag:  *z' 

The  necessary  context-sensitive  constraints  can  be 
superimposed  in  a  natural  manner  by  associating  with  each 
nonterminal  certain  affixes  and  generalizing  the  cfg  rules 
to  enforce  certain  relationships  among  these  affixes. 

A  specific  tag  can  be  remembered  by  generalizing  rules 
r6-r8  as: 

( P 6)  tag  (Tx)  :  'x' 

( P 7)  tag  (Ty)  :  'y' 

( p 8)  tag  (Tz)  :  *z ' 

The  requirement  that  a  tag  be  defined  before  use  is  met 
by  generalizing  r5  as: 

( p  5)  ulist  (fSET):  ulist  (fSET)  'use*  tag  (TTAG) 

is_ member ( fSET, fTAG) 

Here,  "is_member"  is  a  primitive  predicate  whose  purpose  is 
to  make  sure  some  relationship,  defined  elsewhere,  holds 
between  SET  and  TAG.  In  this  case,  is_member  is  used  to 
ensure  that  TAG  is  a  member  of  SET.  The  affix  variable  SET 
carries  the  set  of  "useable”  tags,  i.e.  those  previously 
defined. 

Rule  p5  assumes  that  the  inherited  affix  of  ulist 
carries  the  useable  tag  set.  Rule  rl  is  generalized  to  make 
this  so.  The  nonterminal  dlist  is  given  a  derived  affix 
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that  represents  the  set  of  tags  defined  in  its  terminal 
production.  This  set  is  passed  on  to  the  inherited  affix  of 
ulist . 

(pi)  data:  'begin*  dlist  (TSET)  ulist  (fSET)  'end* 

The  set  of  defined  tags  is  initialized  by  generalizing 

rule  r2.  The  primitive  predicate  "del’',  defined  elsewhere, 

is  introduced  into  the  generalized  rule  p3  to  perform  two 

duties:  it  updates  the  defined  tag  set  for  each  newly 

defined  tag  while  at  the  same  time  enforcing  the  restriction 

against  multiple  definitions.  That  is,  del  ensures  that 

SET  =  SET  1  0  (TAG)  and  TAG  not  in  SET1. 

( p  3)  dlist  (TSET)  :  dlist(TSETI)  'define*  tag  (TTAG) 

del  (tSETI , tTAGTlSETf 

Putting  things  together,  we  get  the  following  affix 
grammar. 

Domains:  Tag  =  { x , y , z}  ;  Tagset  =  power set  of  Tag 

Terminals:  begin,  define,  use,  end,  x,  y,  z 
Nonterminals : 

data,  dlist  (TTagset)  ,  ulist  ( fTagset)  ,  tag(TTags) 
Primitive  Predicates : 

del (fSI : Tagset ,  fT:Tag,  TS2:Tagset)  = 

(T  not  in  SI)  and  (S2  =  SI  0  {T} ) 
is_member (fS: Tagset ,  fT:Tag)  -  (T  in  S) 

Affix-Variables:  TAG:  Tag;  SET,  SET1 :  Tagset 

Rules : 

(pi)  data:  'begin'  dlist (TSET)  ulist  (fSET)  'end' 

( p  2)  dlist  (TSET)  :  'define'  tag(TTAG)  del  ( f  {}  ,  fTAG,  f  SET) 
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( p 3)  dlist(TSET):  dlist(TSETI)  'define*  tag  (TTAG) 

del  (fSETI ,  tTAG7tSET) 

(p4)  ulist  (fSET)  :  e 

(p5)  ulist  (f  SET)  :  ulist  (fSET)  'use*  tag  (TTAG) 

is_member ( fSET7tTAG) 

(p6)  tag(Tx):  'x* 

(p7)  tag  (Ty)  :  'y  ' 

( P 8)  tag  (Tz)  :  'z  ' 

The  ag  rules  are  applied  in  the  following  manner.  Each 

affix- variable  (such  as  SET  or  TAG)  occurring  in  a  given 

rule  is  systematically  replaced  by  a  suitable  affix.  For 

example,  replacing  SET  by  (x,y}  and  TAG  by  x  in  rule  p5,  we 

obtain  the  grammar  rule 

ulist  (f  (x , y} )  :  ulist  (f  (x,  y} )  'use*  tag(Tx) 

is__member  ( t  {x  ,  y)  ,  fx) 

(Watt  uses  the  term  "production  rule"  instead  of  "grammar 
rule".  The  term  "production  rule",  however,  carries  an 
unwanted  active  connotation,  so  we  have  substituted  "grammar 
rule".)  The  objects  **  ulist  (f  (x,  y} ) "  and  "tag(Tx)"  are 
examples  of  notions  ”  each  consists  of  a  nonterminal  with 
its  associated  affixes.  Notions  are  generalizations  of  cfg 
nonterminals  and  are  analogous  to  attributed  symbols  in  an 
atg.  Each  derivation  tree  in  the  ag  will  have  terminals  at 
its  leaves  and  notions  at  its  other  nodes. 

If  the  required  relationships  among  the  affixes  of  a 
primitive  predicate  are  not  satisfied,  we  call  this  a  blind 
a lley .  A  blind  alley  always  blocks  a  derivation  in  which  it 
occurs.  For  example,  "is_member (t {x} , ty) "  is  a  blind  alley, 
thus  enforcing  the  definition-before-use  context-sensitive 
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constraint . 


As  another  example,  the  syntax  of  the  record-group 
language,  including  context-sensitive  constraints,  is  given 
by  the  following  ag  rules. 

data:  group (TKEY)  glist(fKEY) 

glist(fKEY):  group(TKEY2)  uneg (tKEY, tKEY2)  glist(fKEY2) 

I  § 

group  (TKEY):  record  (TKEY)  rlist  (tKEY) 

r  list  (fKEY)  :  record  (TKEY2 )  eg  (fKEY,  fKEY2)  rlist  (fKE  Y2) 

I  § 

where  the  primitive  predicate  "eg”  is  true  if  and  only  if 
its  affixes  are  egual,  and  "uneg"  is  true  if  and  only  if  its 
affixes  differ,  i.e.  uneg  =  non  eg. 

4.3.2  Definition  of  Affix  Grammars 

The  original  affix  grammar  definition  was  given  in 
[  Koster  71],  The  defintion  given  here,  due  to  Watt  [74],  is 
simplified  in  that  it  makes  no  commitment  to  a  particular 
parsing  method. 

Formally,  an  affix  grammar  is  an  8-tuple 
G=  (Vn, Vt ,Q,S , B,D,C, P)  whose  elements  are  defined  as  follows. 

Vn,  Vt  and  Q  are  disjoint  sets  of  nonterminal, 
terminal,  and  primitive  predicate  symbols.  respectively. 
S  6  Vn  is  the  start  symbol. 

B  is  the  set  of  affix- variables.  D  is  a  collection  of 
sets  Db,  one  for  each  affix  variable  b,  which  are  the 
domains  of  the  affix- variables.  Each  object  in  a  domain  Db 
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is  an  affix. 

C  is  the  control  of  G,  a  collection  of  5-tuples 
Cx=  (x  ,  fix,  Nx,  Ax,Fx)  ,  one  for  each  nonterminal  and  primitive 
predicate  symbol  x,  whose  elements  are  as  follows: 

(a)  Mx  >  0  is  the  number  of  inherited  affix -positions  of 
x 

(b)  Nx  >  0  is  the  number  of  derived  affix -posit ions  of  x 

(c)  Ax  is  an  (Mx* Nx) -tuple  of  sets  of  affixes,  which  are 
the  domains  of  the  affix- positions  of  x 

(d)  Fx  is  the  associated  function  of  x,  defined  only  if 
x  is  a  primitive  predicate  symbol.  It  is  a  total 
recursive  Boolean  function  over  the  affixes  of  x: 

Fx:  Ax[  1  ]  x  Ax[2]  x  ...  x  A  x[  Mx*N  x  ]  ->  {false, true} 

Before  defining  P,  we  introduce  an  auxiliary 

definition.  A  hypernotion  is  an  object  of  the  form 

X  (tf[  1  ],  •  . .  *  tf[  Mx  ],Tg[  1  ], . . .  ,  Tg[  Nx  ])  ,  where  x  is  in  Vn  U  Q 

and  each  f[  i  ]  and  g[i]  is  either  an  affix  or  an  affix- 

variable.  x  is  called  the  head  of  the  hypernotion. 

P  is  a  finite  set  of  rules,  each  of  the  form 

Z:  Z1  Z2  ...  Zr  where  Z  is  a  hypernotion  and  each  Zi  is 
either  a  hypernotion  or  a  terminal. 

A  rule  may  be  used  for  generating  cf-like  grammar 

rules,  which  may  in  turn  be  used  for  generating  sentences  in 
the  ag.  Let  v1,v2,...,vn  be  all  the  af f ix-variables 
occurring  in  the  rule  Z:  Zi  ...  Zr.  Let  a1,a2,...,an  be 
arbitrary  affixes  in  the  respective  domains  of  vl , v2 , . . . , vn . 
If  the  result  of  systematically  substituting  a1,a2,...,an 
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for. 

respectively,  v1,v2,.... 

vn  in 

tha  t 

rule 

is 

Y:  Y 1 

...  Yr,  then  the  latter 

is  a 

grammar 

rule 

and 

Y  1  .  .  . 

Yr  is  a  direct  production 

of  Y. 

A  protonotion  is  a  hypernotion  in  which  each  affix 
variable  has  been  replaced  by  an  affix  in  the  domain  of  that 
variable.  Depending  on  the  value  of  its  associated  function 
applied  to  its  affixes,  a  protonotion  whose  head  is  a 
primitive  predicate  symbol  either  has  exactly  one  direct 
production  -  e  -  or  none:  e  is  a  direct  production  of 
q  (tf[  1  ],.  .  . ,  tf[Mq  ],Tg[  1  ],. . .  ,Tg[  Nq])  if  and  only  if 
Fg  (f[  1  ]#•../ f[  Mq  ]*g[  1  ],...  ,g[  Nq  ])  =  true. 

A  protonotion  that  has  at  least  one  direct  production 
is  called  a  notion.  A  protonotion  that  has  no  direct 
production  is  called  a  blind  alle y. 

A  sentence  of  G  is  a  production  of  S  that  contains  only 
terminals.  The  language  generated  by  G  is  the  set  of  all 
sentences  of  G. 

The  unlerlling  translation  grammar  of  an  ag  is  the 
translation  grammar  obtained  by  stripping  the  ag  of  all  its 
affixes  and  a f f ix-var iables.  -The  underlying  cfg  of  an  ag  is 
the  cfg  obtained  by  stripping  the  ag  of  all  its  affixes, 
a f fix- variables,  and  primitive  predicates  symbols.  Note 
that  an  ag's  underlying  cfg  is  identical  to  the  input 
grammar  of  its  underlying  translation  grammar. 

4.3.3  Well-Formed  Affix  Grammars 

In  order  to  make  ag*s  more  suitable  for  one  pass,  left 
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some  conditions  are  imposed  on  the 


to  right  parsing, 
grammar.  But  first  we  need  some  defintions. 

A  defining  affix- £0 si t i on  in  a  rule  is  an  inherited 
affix-position  of  the  left-side  hypernotion,  or  a  derived 
affix  position  of  a  right-side  hypernotion.  An  applied 
affix- position  in  a  rule  is  an  inherited  affix- position  of  a 
right-side  hypernotion,  or  a  derived  affix-position  of  the 
left-side  hypernotion. 

A  well-formed  affix  grammar  is  an  ag  that  satisfies  the 
following  three  conditions  [Watt  77a]: 

(1)  Every  defining  affix-position  in  a  rule  is  occupied 
by  an  affix-variable  whose  domain  subsumes  the 
domain  of  the  affix-position;  and  every  applied 
affix-position  is  occupied  either  by  an  affix- 
variable  whose  domain  is  a  subset  of  the  domain  of 
the  affix-position  or  by  an  affix  in  the  domain  of 
the  affix-position. 

(2)  For  each  primitive  predicate  symbol  q,  there  exists 
a  partial  recursive  function  mapping  its  inherited 
affixes  onto  its  derived  affixes: 

Fq1:  Aq[  1  ]  X  ...  X  Aq[Mq]  -> 

Aq[Mq+1]  x  ...  x  Aq[Mq+Nq]  such  that 
Fq*  (a[  1  ],...,  a[  Mq  ])  =  (a[  Mq+ 1  ],...,  a[  Mq +N q  ]) 

if  and  only  if  Fq  (a[  1  ],..., a[  Mq  +  Nq  ])  is  true. 

(3)  Every  affix-variable  occurring  in  a  rule 
Z:  Z 1  ...  Zr  occurs  in  exactly  one  defining  affix- 
position  in  that  rule.  Moreover,  if  its  defining 
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occurrence  is  in  a  hypernotion  Zj  in  the  right  side 
of  the  rule,  then  it  occurs  in  no  applied  position 
in  any  of  Z1  ...  z j . 

Condition  1  ensures  that,  during  parsing,  every  affix 
is  within  the  domain  of  the  affix-position  it  occupies. 

Condition  2  allows  the  inherited  and  derived  affixes  of 
a  primitive  predicate  to  be  interpreted  as  input  and  output 
parameters,  respectively,  of  its  associated  function. 
Moreover,  if  the  function  F’  is  not  total,  it  has  the 
additional  effect  of  imposing  a  context-sensitive 
constraint. 

The  first  part  of  condition  3  ensures  that,  during 
parsing,  every  affix-variable  is  defined,  and  never  re¬ 
defined  (which  would  violate  the  systematic  substitution 
rule) .  Moreover,  since  each  inherited  affix  of  the  left¬ 
side  notion  uniquely  defines  an  affix-variable,  as  does  each 
derived  affix  of  each  right-side  hypernotion,  it  can  be  seen 
that  inherited  affixes  propagate  "down",  and  derived  affixes 
•"up" ,  the  parse  tree. 

The  second  part  of  condition  3  ensures  that  each  affix- 
variable  is  available  when  required  during  a  single  left  to 
right  pass  over  the  syntax  tree. 

The  choice  of  the  term  "well-formed  ag"  is  unfortunate 
because  well-formed  ag’s  do  not  correspond  to  similarly- 
named  well-defined  atg's.  Rather,  the  well-formed  criteria 
for  ag’s  are  equivalent  to  the  L-attributed  criteria  for 
atg’s.  Thus,  remarks  made  about  L-atg’s  apply  equally  well 
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to  well-formed  ag's,  and  vice  versa.  In  particular,  it 
means  that  inherited  affixes  (inherited  attributes)  carry 
information  about  left  context  and  derived  affixes 
(synthesized  attributes)  carry  information  about  a  produced 
phrase  in  the  left  context.  Similarly,  the  ag  parsing 
algorithm  to  be  presented  in  the  next  section  applies  as 
well  to  L-atg's.  This  is  significant  because  it  handles  a 
much  wider  class  of  grammars  than  the  L-atg  subclasses  of 
Lewis  et  al. 

As  in  L-atg's,  regularized  grammar  rules  are  not 
permitted  in  well-formed  ag's.  Using  the  ag  terminology,  we 
can  be  more  precise  as  to  why:  a  grammar  rule  with  a 
derived  affix-position  within  a  ♦-subexpression  would 
violate  the  rule  against  a f fix- variable  redefinition 
(condition  3)  . 

4.3.4  Parsing  Well-Formed  Affix  Grammars 

In  this  section  we  look  at  parsing  methods  for  well- 
formed  ag's.  We  concentrate  on  the  single  pass  method 
developed  by  Watt  [  74,  77a]. 

Watt's  parsing  method  is  based  on  the  following  fact: 
in  a  well-formed  ag,  the  inherited  affixes  of  a  nonterminal 
notion  are  known  before,  and  the  derived  affixes  after, 
scanning  the  phrase  produced  from  the  nonterminal. 

His  parsing  strategy  is  as  follows.  Immediately  before 
scanning  a  phrase  that  may  be  a  production  of  some  notion, 
the  inherited  affixes  of  the  notion  are  pushed  onto  a 
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special  affix  stack,  unless  they  are  already  present  at  the 


top  of  the  stack.  After  scanning  the  phrase,  the  derived 
affixes  of  the  notion  are  determined  and  stacked  immediately 
above  the  inherited  affixes,  replacing  any  affixes  that  may 
have  been  put  there  during  the  parsing  of  the  phrase.  In 
this  way,  the  value  of  each  affix-variable  can  always  be 
found,  when  required,  at  a  known  position  relative  to  the 
top  of  the  affix  stack. 

The  ag  parser  is  basically  a  cfg  parser  extended  to 
perform  some  manipulation  of  the  affix  stack  upon  each 
reduction.  The  parser  is  constructed  from  a  translation 
grammar  known  as  the  head  grammar  determined  by  the  ag.  To 
obtain  the  head  grammar,  we  start  with  the  underlying 
translation  grammar  of  the  ag.  Then,  special  action  symbols 
called  copy  symbols  are  inserted  at  appropriate  places  in 
the  grammar  rules  of  the  head  grammar.  In  general,  whenever 
the  inherited  a f fix- positions  of  a  right  side  hypernotion 
are  occupied  by  affixes  and  a f fix- variables  that  do  not 
exactly  match,  in  order,  the  topmost  items  of  the  affix 
stack  configuration,  a  copy  symbol  is  inserted  in  the 
grammar  rule.  Each  copy  symbol  is  of  the  form 
Ccopy  a1,...,am>,  where  m>0  and  each  ai  is  either  an  affix 
or  of  the  form  "stack  d",  where  d  is  a  positive  integer. 
The  recognition  of  a  given  copy  symbol  brings  about  the 
associated  stack  manipulation  as  a  side  effect. 

In  addition,  each  grammar  rule  of  the  head  grammar  has 
an  associated  action  to  be  performed  upon  completion  of  the 
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given  rule.  This  action  ensures  that  the  derived  affixes  of 


the  left  side  notion  are  placed  immediately  above  its 
inherited  affixes  on  the  affix  stack. 

Watt's  method  thus  reduces  the  ag  parsing  for  a  large 
class  of  ag's  to  a  translation  grammar  parsing  problem. 
While  it  is  possible  for  the  designer  to  construct  an 
equivalent  translation  grammar  by  himself,  the  ag  method  is 
more  likely  to  be  both  useful  and  correct,  since 
parameterization  is  part  of  the  notation  and  its 
implementation  is  done  automatically  by  the  constructor. 

We  now  develop  some  ag  parsing  results.  An  ag  is 
defined  to  be  AF-LR  (k)  (AF-LL  (k)  )  if  and  only  if  it  is  well- 
formed  and  its  head  grammar  is  LR(k)  (LL ( k) ) •  (Here,  "  A  F  " 
stands  for  "a f f ix- fre e" ,  in  reference  to  the  dependence  on 
the  affix-free  head  grammar.)  The  translation  grammar 
parsing  theorems  can  be  used  to  show: 

(1)  an  ag  is  AF-LL  (k)  if  and  only  if  its  underlying  cfg 
is  LL  (k)  . 

(2)  an  ag  is  AF-LR(k)  only  if  its  underlying  cfg  is 


LR  (k)  . 

Note  that  an  ag's  underlying  cfg  is  identical  to  its  head 
grammar's  input  grammar. 

The  novel  feature  of  Watt's  method  is  that  inherited 
affixes  are  pushed  onto  the  affix  stack  only  when  not 
already  present,  in  the  correct  order,  at  the  top  of  the 
affix  stack.  This  is  not  merely  a  minor  optimization  that 
saves  a  few  machine  instructions  in  infrequent  situations; 
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it  can  be  exploited  frequently  in  practice,  and  it  greatly 
extends  the  class  of  ag's  for  which  deterministic  parsers 
can  be  constructed  automatically.  Examples  of  ag's  not 
parsable  deterministically  by  other  current  methods  are 
given  in  [Watt  77a]. 

The  simple  strategy  of  stacking  all  inherited  affixes 
is  the  one  adopted  by  Lewis  et  al.  and  never  causes  trouble 
for  ag's  with  LL  (k )  underlying  cfg's.  However,  this 
strategy  breaks  down  for  bottom-up  parsing  methods,  as  it 
prevents  full  exploitation  of  the  greater  power  of  bottom-up 
techniques.  For  this  reason,  Lewis  et  al.  consider  only 
LR(k)  postfix  L-atg's  with  inherited  attributes.  It  is  in 
the  bottom-up  area  where  Watt's  method  makes  tremendous 
gains . 

AF-LR  parsers  have  the  following  limitations:  affixes 
can  never  influence  the  flow  of  control  through  the  parser, 
other  than  making  it  halt  on  a  context-sensitive  error  when 
some  constraint  is  not  met.  A  natural  extension  of  this 
parsing  method  would  be  to  give  affixes  the  ability  to 
influence  flow  of  control.  This  extension  would  allow 
deterministic  parsing  of  unambiguous  ag's  with  ambiguous 
underlying  cfg's.  Of  course,  certain  conditions  must  be 
imposed  on  the  grammar  and  predicates  to  ensure 
deterministic  parsing.  A  discussion  of  a  sufficient  set  of 
conditions  is  given  in  [Watt  74]. 

Multipass  parsers  along  the  lines  of  Bochmann's  method 
are  also  conceivable.  Applied  to  ag's,  Bochmann's  technique 
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would  allow  a  considerable  relaxation  of  the  second  part  of 
condition  3  for  well-formedness.  Research  is  currently 
being  conducted  on  a  combination  of  Bochmann's  technique 
with  Watt's,  which  would  allow  efficient  multipass  analyzers 
to  be  constructed  automatically  [Watt  77b]. 
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CHAPTER  5  -  A  PROPOSAL 


In  this  chapter  we  present  a  grammatical  formalism 
intended  for  programming  that  satisfies  the  criteria 
established  in  Chapters  3  and  4.  A  grammar  in  our  formalism 
is  called  an  affix  grammar  for  programming  (agp)  .  In 
developing  the  agp  formalism,  we  have  taken  well-formed  ag's 
and  modified  them  to  make  them  more  suitable  for 
programming.  The  most  significant  modifications  are  meant 
to  put  the  specification  of  data  and  actions  in  a  unified 
framework  and  to  take  further  advantage  of  the  lef t-to-right 
orientation  of  well-formed  ag's. 

5.1  Grammatical  Symbols 

In  a  well-formed  ag  there  are  three  types  of 
grammatical  symbols:  nonterminals,  terminals,  and  primitive 
predicates.  The  first  two  each  have  a  single  role:  a 
nonterminal  symbol  represents  a  structure  component  and  a 
terminal  symbol  specifies  an  atomic  data  item.  A  primitive 
predicate,  however,  serves  a  dual  purpose:  it  specifies 
both  a  relational  constraint  and  an  input/output 
relationship  on  its  affix-positions.  Each  primitive 
predicate  symbol  x  has  two  associated  functions,  a  total 
recursive  function  Fx  that  maps  the  value  space  of  its 
affix- positions  into  {false, true}  and  a  partial  recursive 
function  Fx',  determined  from  Fx,  that  maps  the  value  space 
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of  its  inherited  aff i x- positions  into  the  value  space  of  its 
derived  ones.  Fx  serves  to  impose  a  constraint,  while  Fx1 
determines  an  action. 

In  an  agp  we  take  advantage  of  the  fact  that  the  same 
structuring  mechanisms  can  be  used  to  structure  both  data 
and  actions.  This  allows  both  data  and  actions  to  be 
refined  together  within  the  grammar  rules  in  a  unified 
manner . 

We  formalize  the  intuitive  notion  of  an  action  as 


follows.  An  action 

binds  a 

set 

of  values. 

wh 

ich 

has  been 

determined  from  a 

set  of 

input 

parameters 

r 

to 

a  set  of 

output  variables. 

That  is. 

if 

i  is  an  m- 

tu 

pie 

of  input 

parameters  for  a  given  action,  o  is  an  n-tuple  of  distinct 
output  variables,  and  F  is  the  n-valued  function  associated 
with  the  action,  then  the  result  of  the  action  is 
(o[1  ],o[2  ],...,o[n])  :=  F  (i[  1  ],  i[  2  ],...,  i[  m  ]) 

The  definition  makes  no  assumptions  about  sequencing  within 
an  implementation  of  an  action.  It  implies  that  input 
parameters  are  passed  by  value  and  output  parameters  are 
passed  by  result.  This  helps  avoid  some  aliasing  problems 
that  might  otherwise  occur  within  an  action's 
implementation. 

Each  elementary  component  in  an  agp  structure  is  one  of 
the  following:  a  terminal,  a  basic  action,  or  a  Rredicate. 
Elementary  components,  of  all  three  types,  can  be  combined 
together  using  the  structuring  mechanisms  to  form 
composites,  which  in  turn  can  be  components  of  larger 
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composites.  A  grammar  rule  specifies  that  the  nonterminal 
symbol  on  the  left  side  is  to  represent  the  rule's  right 
side  whenever  the  nonterminal  appears  in  the  right  side  of 
any  rule  (including  its  own)  .  A  predicate  in  a  given 
grammar  rule  imposes  a  side-effect- free  Boolean  condition 
that  must  be  met  in  each  application  of  that  rule.  Thus,  in 
an  agp  there  are  four  types  of  grammatical  symbols,  each 
with  a  distinct  role:  nonterminals,  terminals,  basic 
actions,  and  predicates. 

Corresponding  to  affix  lists  in  ag's  are  parameter 
lists  in  agp's.  Nonterminal  and  basic  action  symbols  are 
the  two  types  of  symbols  to  possess  parameter  lists  and  are 
referred  to  as  parameterized  symbols.  In  deference  to  agp's 
ag  heritage,  we  use  the  term  hypernotion  to  denote  a 
parameterized  symbol  paired  with  a  parameter  list.  The 
parameterized  symbol  associated  with  a  given  hypernotion  is 
called  the  hypernotion ' s  head.  The  elements  occupying  a 
parameter  list  are  called  affix-parameters.  which  are 
partitioned  into  sets  of  input  affix-parameters  and  output 
affix-parameters.  These  correspond  to  the  inherited  and 
derived  affix-positions  in  an  affix  grammar. 

We  may  think  of  input  parameters  as  being  passed  by 
value  and  output  parameters  passed  by  result.  The  decision 
to  make  the  sets  of  input  and  output  parameters  disjoint  was 
made  deliberately,  though  the  detailed  reasons  for  this 
decision  are  not  relevant  to  this  discussion. 

When  on  the  left  side  of  a  rule,  the  input  affix- 
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paraaeters  specify  the  variable  names  to  which  the  input 
values  are  to  be  bound  and  the  output  affix-parameters 
supply  output  values.  Similarly,  when  on  the  right  side  of 
a  rule,  the  input  affix-parameters  supply  input  values  and 
the  output  affix-parameters  specify  the  variable  names  to 
which  the  output  values  are  to  be  bound. 

These  ideas  are  captured  by  the  definitions  for 
defining  and  applied  affix-parameters.  Def ini ng  fiarameters 
bind  values  to  variables;  they  are  the  input  parameters  on 
the  left  side  of  a  grammar  rule  and  the  output  ones  on  the 
right.  Applied  parameters  supply  values;  they  are  the 
output  parameters  on  the  left  and  the  input  ones  on  the 
right . 

These  ideas  define  the  role  of  each  type  of  grammatical 
symbol  in  an  agp.  Once  affix-variables  and  affix-parameters 
are  introduced,  we  need  to  discuss  the  idea  of  a  data  type. 
After  presenting  our  view  of  a  data  type  in  the  next 
section,  we  take  a  detailed  look  at  how  predicates  and 
actions  are  used  in  an  agp.  A  discussion  of  the  formal 
properties  of  well-formed  agp's,  including  the  strong  left- 
to-right  orientation,  completes  the  presentation  of  agp 
fundamentals,  and  we  conclude  the  chapter  with  a  short 
discussion  of  inter-agp  communication.  A  presentation  of 
agp  notation  and  agp  examples  follows  in  Chapter  6. 

5.2  Data  Types 

Before  discussing  data  types,  we  must  emphasize  that 
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data  types  are  not  a  central  concern  of  ours  in  agp's,  and, 
as  such,  we  are  regrettably  imprecise  at  times.  In  agp's  we 
assume  that  some  sufficiently  rich  set  of  built-in  data 
types  is  provided.  A  facility  to  construct  user-defined 
data  types  is  a  nice  convenience,  but  not  inherent  to  the 
agp  approach. 

A  data  txpe  consists  of  a  set  of  values  called  the 
type's  range  and  a  set  of  operations  on  those  values  that 
completely  characterize  the  behavior  of  the  values  in  the 
type.  A  variable  represents  a  value  in  the  range  of  a  given 
data  type. 

In  our  view  of  data  types,  each  type  operation  is 
strictly  functional:  it  produces  values  without  affecting 
its  arguments.  In  this  view,  constants  are  parameterless 
functions.  With  respect  to  arrays,  for  example,  this 
functional  view  means  that  an  array  is  viewed  as  a  compound 
value,  rather  than  a  collection  of  cells.  The  expression 
"A[ij"  denotes  the  i'th  value  in  the  array  A;  it  does  not 
denote  a  cell  and  cannot  be  used  on  the  left  of  an 
assignment.  The  expression  "  (A;i:x)"  denotes  an  array  just 
like  A,  except  that  its  i'th  value  is  x. 

We  often  find  it  notationally  convenient  to  write 
functions  involving  the  familiar  operations  in  an  infix 
operator  notation  with  the  usual  operator  precedence  and 
associativity  relations.  For  example,  we  may  write 
"add  (x, mult  (y ,z) ) "  as  "x+y*z"  and  "and  (It  (i,  j)  ,lt  ( j,k)  )  "  as 
"i<j  and  j<k"  . 
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When  using  a  data  type  in  Chapter  6,  we  give  the 
functionality  of  a  type's  operations  and  assume  some 
implementation  is  provided.  We  do  not  want  to  get  bogged 
down  with  data  type  implementation  details,  for  they  would 
only  detract  from  the  presentation  of  the  unique  features  of 
agp' s. 

5 • 3  Predicates  and  Actions 

A  predicate  symbol  represents  a  Boolean  function.  A 
given  instance  of  a  predicate  symbol  can  be  used  in  a 
derivation  if  and  only  if  its  associated  Boolean  function 
evaluates  to  true.  Thus,  a  predicate  helps  determine  which 
instance  of  a  rule,  if  any,  is  to  be  used  in  a  derivation. 
Predicate  symbols  in  an  agp  are  true  to  their  name:  they 
represent  side-effect-free  predicates  and  not  actions. 

A  basic  action  symbol  represents  an  action  that  cannot 
be  refined.  Onlike  predicates,  basic  actions  do  not  impose 
constraints.  At  the  very  least,  we  need  basic  actions  to 
perform  typed  assignments.  For  each  data  type  there  is  then 
an  assignment  hypernotion  with  two  affix-parameters,  an 
input  (source)  and  an  output  (destination)  .  The  source  is  a 
function;  the  destination  is  an  affix- variab le.  A  given 
assignment  action  simply  binds  the  value  of  its  source  to 
its  destination.  Higher-level  actions  can  be  specified  in 
the  grammar  rules  by  structuring  predicates,  assignments, 
and  other  action  components. 

As  a  notational  convenience  we  use  a  single  generic 
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assignment  hypernotion,  denoted  by  the  infix  symbol 
rather  than  separate  ones  for  each  type.  The  left  operand 
of  "  is  the  destination;  its  right  operand  is  the  source. 
The  action  is  well-defined  if  and  only  if  the  source  is 
assignment-compatible  with  the  destination. 

For  this  minimal  approach  to  suffice,  it  is  essential 
that  predicates  be  fully  capable  of  influencing  the  parser's 
flow.  If  the  parser  lacks  this  capability  (as  in  an  AF-LR 
parser) ,  we  must  view  a  basic  action  symbol  as  the  name  of  a 
separately  defined  action  routine  whose  input/output 
interface  is  defined  by  the  symbol's  parameter  list.  In 
this  case,  the  use  of  basic  actions  is  similar  to  the 
original  use  of  primitive  predicates,  except  that  the 
imposition  of  constraints  is  still  left  to  agp  predicates. 

An  example  is  presented  to  contrast  our  approach  to 
actions  and  predicates  with  that  of  ag's.  Let  us  reconsider 
the  ag  primitive  predicate  "del  (SI , T, S2) "  from  the  ag 
example  is  Section  4.3.1.  Si  and  S2  are  Tagsets  and  T  is  a 
Tag.  "del",  defined  outside  of  the  ag  grammar  rules, 
represents  both  a  predicate  and  an  action:  the  predicate 

ensures  that  no  tag  be  declared  twice,  i.e.,  that 
"T  not  in  Si"  holds;  the  action  adds  T  to  the  set  of  defined 
tags,  i.e.,  "S2  :=  SI  D  {T}".  In  an  agp,  we  would 

distinguish  between  the  constraint  and  the  action  and  place 
them  directly  in  the  grammar  rules.  Thus,  wherever 

"del  (S1,T,S2) "  appears,  we  would  substitute  "T  not  in  SI 
S2:=S1  U  {T}",  or  the  equivalent  in  whatever  notation  is 
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provided  for  set  operations.  The  important  point  is  that  in 
an  agp,  conditions  and  actions  are  distinct  and  explicit. 

Program  output  is  accomplished  by  executing  an  output 
hypernotion.  Since  we  do  not  wish  to  get  bogged  down  with 
output  considerations,  we  introduce  only  two  simple  output 
basic  actions.  The  two  output  hypernotions  are 
" print  (vl , v2 ,..., vn) "  and  "print  In (vl , v2, ..., vn ) "  where  n>0 
and  may  vary  in  individual  instances.  The  n  affix- 
parameters  of  an  output  hypernotion  are  all  input 
parameters.  When  executed,  "print  (vl ,..., v2) "  outputs  the 
values  of  v1,...,vn.  "printin’1  performs  a  similar  task, 
except  that  it  starts  a  new  output  line. 

5.4  Properties  of  Well- Formed  Agp ’ s 

Before  presenting  some  formal  properties  of  well-formed 
agp’s,  we  present  these  properties  informally,  starting  with 
with  strong  lef t-to-r ight  orientation. 

Condition  3  of  the  well-formed  ag  definition  in  Section 
4.3  imposes  a  distinct  left- to-right  orientation  on  well- 
formed  ag  rules.  When  we  noted  this  same  lef t-to-righ t 
orientation  in  L-atgfs  (well-formed  ag*s  atg  analogue),  we 
asserted  that  it  was  guite  appropriate  for  programming.  In 
agp • s,  we  take  this  lef t- to- righ t  orientation  a  step  further 
than  do  either  well-formed  ag's  or  L-atg's:  we  allow  an 
aff ix-variable  to  be  redefined  within  an  agp  rule.  The 
value  of  a  particular  applied  occurrence  of  an  affix- 
variable  is  simply  the  one  most  recently  defined. 
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Obviously,  a  variable  must  be  defined  before  it  can  be 
applied.  Potential  aliasing  problems  are  avoided  by 
imposing  the  restriction  that  a  given  aff ix-variable  may  not 
appear  in  more  than  one  defining  position  in  a  single 
parameter  list. 

Since  an  applied  affix-parameter  supplies  a  value,  it 
may  in  general  be  an  affix  function,  rather  than  simply  an 
affix-variable  or  a  constant  as  in  ag*s.  A  special  case  of 
an  affix  function  is  an  identity  function  of  a  given  affix- 
variable. 

However,  there  is  a  perceptual  problem  if  we  allow 
applied  output  parameters  (output  parameters  on  the  left)  to 
be  affix  functions.  The  evaluation  of  these  parameters  is 
the  last  thing  to  be  performed  in  the  recognition  of  a  given 
rule.  Yet,  they  appear  notationally  at  the  left  end  of  the 
rule.  If  a  given  applied  output  position  in  a  rule  is 
occupied  by  the  function  f (vl, v2,. . . , vk) ,  where  each  vi  is 
an  affix- variable  defined  on  the  right  side  of  the  rule, 
then  it  misleadingly  appears  as  though  the  variables  are 
being  applied  before  they  are  defined. 

If  instead  the’  position  is  filled  by  a  placeholder 
a ff ix-variable  v  and  " v: =f  (v 1 ,. , . , vk) M  is  inserted  at  the 
right  end  of  the  rule,  the  problem  is  alleviated.  For  this 
reason,  we  recommend,  but  do  not  restrict,  applied  output 
positions  to  be  filled  by  affix- variable s  rather  than  affix 
functions. 

The  above  ideas  are  formalized  in  the  following  four 
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properties  of  a  well-formed  agp.  Properties  1 ,2  and  4  are 
conditions,  while  property  3  is  definitional.  The  first  two 
properties  ensure  uniqueness  of  definition  and  definition 
before  use,  respectively.  The  third  property  imposes  the 
”most  recently  defined”  rule  for  applied  occurrences  of 
a ffix- variables.  The  fourth  property  ensures  assignment 
compatibility  between  applied  and  defining  affix- parameters. 

For  all  properties,  let  ZO  :  Z1  Z2  ...  Zr  be  a  rule. 

(1)  If  the  affix- variable  X  occupies  a  defining  position 
in  Zi  0<i<r,  then  it  occupies  exactly  one  defining 
position  in  Zi. 

(2)  If  the  a ff ix- variable  X  occurs  in  an  applied 
position  in  Zi  0<i<r,  then  it  occupies  a  defining 
position  in  Zj  0<j<r.  Furthermore,  if  i>1  (i.e.,  if 
Zi  is  on  the  right  side) ,  then  j<i. 

(3)  If  the  affix- variable  X  occurs  in  an  applied 
position  in  Zi  0<i<r,  then 

(a)  If  1<i<r,  then  the  value  of  X  in  its  applied 
position  in  Zi  is  given  by  its  defining  position 
value  in  Zj  where  j  is  such  that  0<j<i  and  X 
occupies  no  defining  position  in  Zk  for  j<k<i. 

(b)  If  i=0,  then  the  value  of  X  in  its  applied 
position  in  ZO  is  given  by  its  defining  position 
value  in  Zj  where  j  is  such  that  0<j<r  and  X 
occupies  no  defining  position  in  Zk  for  j<k<r. 

(4)  If  Z  is  the  head  of  the  hypernotion  Zi  0<i<r,  then 
(a)  If  1<i<r,  the  affix  function  f  is  a  given  input 
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parameter  in  the  parameter  list  of  Zi , 


and  the 


affix-variable  X  is  the  input  parameter 

occupying  the  corresponding  (defining)  position 
in  the  parameter  list  of  the  left-side 
hypernotion  with  head  Z ,  then  f  is  assignment- 
compatible  with  X. 

(b)  If  i=0,  the  affix  function  f  is  a  given  output 
parameter  in  the  parameter  list  of  ZO,  and  the 
affix-variable  X  is  the  output  parameter 

occupying  the  corresponding  (defining)  position 
in  the  parameter  list  of  a  right-side 
hypernotion  with  head  Z,  then  f  is  assignment- 
compatible  with  X. 

As  we  saw  in  Section  4.3,  a  grammar  rule  with  a  derived 
affix-position  in  a  *-subexpr ession  violates  the  well-formed 
affix  grammar  restriction  against  an  affix-variable 
occupying  more  than  one  defining  position.  For  this  reason, 
regularized  rules  were  not  permitted  in  affix  grammars. 
However,  in  an  agp  this  restriction  does  not  apply  (property 
3).  Hence,  the  more  flexible  regularized  style  is  adopted 
for  agp  grammar  rules.  While  regularized  rules  are  not 
essential,  we  have  seen  that  they  can  be  very  useful. 

With  regularized  rules,  properties  2  and  3  must  be 
rephrased  in  terms  of  rule  instances.  Even  though  a  rule 
that  contains  a  ^-subexpression  has  an  infinity  of 
instances,  it  is  necessary  to  consider  only  a  finite  number 
of  cases  in  order  to  determine  whether  property  2  holds  for 
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all  instances  of  that  rule.  Specifically,  if  a*  is  a 
subexpression  in  the  RE  b  and  A:  b  is  a  rule,  then  the  only 
instances  of  a*  within  instances  of  b  we  need  consider  are 
1(e)  and  1(a)  in  order  to  determine  whether  for  all 
instances  of  b,  all  applied  occurrences  of  af f ix-variables 
within  a  are  defined  before  use. 

Since  we  will  deal  only  with  well-formed  agp's,  the 
qualifier  "well-formed”  will  be  assumed. 

5.5  Inter-Agp  Communication 

In  Section  3.3  we  demonstrated  tokenizing  to  be  a  very 
general  framework  for  inter-modular  communication.  The 
support  of  tokenizing  in  agp’s  requires  the  introduction  of 
a  mechanism  to  pass  tokens  between  modules. 

A  token  in  an  agp  is  an  instance  of  a  terminal  or  an 
instance  of  a  nonterminal  hypernotion  whose  head  does  not 
appear  on  the  left  side  of  a  grammar  rule  in  that  agp.  Each 
different  nonterminal  token  defines  a  separate  token  class 
while  the  set  of  terminals  defines  a  single  token  class.  A 
nonterminal  token  may  possess  affix-parameters,  all  of  which 
must  be  derived.  (This  induces  the  feedback- free  property.) 
These  parameters  are  used  to  carry  information  about  the 
tokenized  component  described. 

The  consumption  of  tokens  in  an  agp  is  implicit:  the 
occurrence  of  a  given  token  symbol  in  an  agp  specifies  that 
a  token  of  that  class  is  the  be  consumed.  Since  the 
emission  of  tokens  is  not  implicit,  the  basic  action  "emit" 
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is  provided.  Emit  possesses  a  variable  number  of  affix- 
parameters,  all  of  which  are  input  parameters.  Each 
parameter  specifies  a  token  class  name  with  optional  values 
in  parentheses.  For  a  nonterminal  token  class,  these  values 
are  assigned  to  the  output  parameters  of  the  nonterminal 
token  in  the  target  agp.  For  example,  in 

"emit  (Constant (i) )  " ,  the  value  of  i  would  be  assigned  to  the 
output  parameter  of  the  token  Constant  in  the  target  agp. 
For  the  terminal  token  class,  the  single  value  must  be  a 
character  string  and  is  the  value  of  a  terminal  symbol  in 
the  target  agp,  e.g.,  emit (Termi nal ( • abc • ) ) . 

The  implementations  of  a  few  often-used  token  classes 
are  built-in.  These  built-in  token  classes  include: 

(1)  char  (c) ,  which  describes  a  character  and  returns  the 
described  character  in  c, 

(2)  letter(l),  which  describes  a  letter  and  returns  the 
described  letter  in  1,  and 

(3)  digit  (d)  ,  which  describes  a  digit  and  returns  the 
described  digit  in  d. 

Of  course,  in  an  implementation,  other  built-in  token 
classes  could  be  supported. 
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CHAPTER  6  -  AGP  NOTATION  AND  EXAMPLES 

In  this  chapter  we  informally  outline  the  agp  notation 
and  give  several  examples  of  agp  programs.  The  notation 
chosen  is  meant  to  be  illustrative  rather  than  definitive. 
Before  an  agp  can  be  considered  to  be  an  actual  program,  a 
more  precise  and  complete  definition  of  agp  syntax  is 
required. 

6 . 1  Agp  Notation 

An  agp  contains  three  sections,  each  of  which  specifies 
a  class  of  definitions:  the  grammar  rules,  data  type,  and 
predicate  sections.  The  grammar  rules  section  contains  the 
agp  grammar  rules.  On  the  left  side  of  a  rule  is  its  left 
side  nonterminal  hypernotion.  The  nonterminal  on  the  left 
side  of  the  first  rule  is  by  convention  the  start  symbol. 
On  the  right  side  of  a  rule  is  a  list  of  local  variable 
declarations  followed  by  a  regular  expression  over  the  set 
of  nonterminals,  basic  actions,  predicates,  and  terminals. 
Each  rule  is  terminated  with  a  period.  The  form  for  a  rule 
is  then: 

Rule:  LhsNontHy p  »:•  Loca lVblDecls  RegExpr  . 

Within  the  parameter  list  of  a  left-side  hypernotion 
are  two  lists:  the  list  of  declarations  for  the  input 
parameter  variables  and  the  list  of  output  parameter  affix- 
functions.  The  two  lists  are  headed  by  a  down  arrow  ("t") 
and  an  up  arrow  ("T")  ,  respectively,  and  are  separated  by  a 
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semicolon.  The  form  for  a  left  side  nonterminal  hypernotion 
is  thus: 

LhsNontHyp:  NontSymbol  (  •  (•  LhsParmList  •)•  )?  . 

LhsParmList:  'f*  VblDecls  •;*  *  T  *  FnList 

|  'f*  VblDecls 

|  •t'  FnList  . 

VblDecls:  Vbls  •  :•  type  (’,'  Vbls  '  type) *  . 

Vbls:  identifier  (  ' identifier  )*  . 

FnList:  AffixFn  (  * , 1  AffixFn  )*  . 

For  example,  "X  Iti . 1: intege r.x : real ;  Ti  +  j/X)n  might  be  a 
left  side  nonterminal  hypernotion. 

The  aff ix-variables  that  occur  on  the  right  side  of  a 
given  rule  but  not  in  the  input  parameter  declarations  are 
local  variables  for  that  rule  and  are  declared  within  the 
rule.  This  is  in  contrast  to  the  ag  case,  where  even  though 
rules  have  local  scoping,  all  affix-variables  are  declared 
globally.  The  agp  approach  is  thus  more  consistent  and  more 
flexible.  The  form  for  the  optional  local  variable 
declarations  is: 

LocalVblDecls:  (*var*  VblDecls  ’;*)?  . 

A  right  side  parameter  list  contains  two  lists 
separated  by  a  semicolon:  the  list  of  affix-function  input 
parameters  and  the  list  of  output  parameter  variables.  The 
input  list  may  optionally  be  headed  by  "t".  Similarly,  the 
output  list  may  optionally  be  headed  by  "T". 

The  data  type  section  contains  the  encapsulated 
implementations  for  user-defined  data  types.  The  details  of 
this  section  depend  on  the  user-defined  type  definition 
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facility,  and,  as  such,  are  not  dealt  with  here. 

The  predicate  section  contains  the  rules  for  predicates 
that  are  not  Boolean  data  type  operations  (either  built-in 
or  user-defined) .  Predicate  rules  are  slightly  different 
from  grammar  rules.  First,  a  "returns (identifier: Boolean) " 
clause  is  inserted  after  the  parameter  list  so  that  the  rule 
can  return  a  functional  Boolean  value.  Second,  no  terminals 
may  appear  on  the  right  side  of  any  rule  in  the  predicate 
section.  Third,  since  all  parameters  are  input  parameters, 
the  Mt”  is  omitted  from  the  input  parameter  declaration 
list . 


6.1.1  Alternative  RE  Notations 

The  notation  we  have  used  f 

(a)  a  1  a2  ...  an  for 

(b)  a*  for 

(c)  al  |  a2  |  ...  |  an  for 

A  slight  variation  of  this  notat 

(a)  al  ;  a2  ;  ...  ;  an 

(b)  *a 

(c)  al  Q  a2  Q  ...  0  in 

Here  semicolons  separate  members 
members  of  an  alternation.  In 
placed  prefix  rather  than  postfi 
operators  (♦ ,  ?,  *n,  +n,  n)  . 

syntax  of  Dijkstra*s  programming 


or  regular  expressions  is: 
sequences , 
iterations,  and 
a  lternations . 
ion  is: 


of  a  sequence  and  boxes  the 
addition,  the  iteration  *  is 
x,  as  are  all  other  unary  RE 
A  notation  related  to  the 
language  [  76  ]  is : 
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(a)  al  ;  a2  ;  ...  ;  an 

( b)  do*  a  od 

(c)  if  al  0  §2  0  •••  D  an  fi 

The  choice  among  the  notations  is  a  matter 
After  trying  all  three  notations  in  a  number  of  e 
decided  (rather  arbitrarily)  on  choice  2.  As 
choice  2  is  the  notation  used  for  RE*s  in  the  ex 
follow. 


6 . 2  Recor d-Gr oup  Language 

In  the  first  example  we  give  the  grammar  ru 
agp's  that  each  describe  the  record-group  lang 
first,  a  recursive  formulation,  is  very  similar 


solution  in  Section  4.3.1. 


The  second,  an 


formulation,  illustrates  the  use  of  *-rules  in 
with  affix-variables  having  multiple  definitions. 

Recall  that  the  language  consists  of 
sequence  of  record  groups.  Each  group  cons 
nonempty  sequence  of  data  records  that  possess  th 
and  two  consecutive  groups  contain  records  th 
different  keys. 

For  the  agp's  below,  we  assume  that  record 
data  type  RecKey.  The  nonterminal  "record”  is 
token  describing  a  data  record.  Its  output  affi 
contains  the  described  record's  key.  The 
formulation  is  as  follows. 


of  taste, 
xamples,  we 
a  result, 
amples  that 


les  for  two 
uage.  The 
to  the  ag 
iterative 
c  on -junction 

a  nonempty 
ists  of  a 
e  same  key, 
at  possess 

keys  are  of 
used  as  a 
x- paramete  r 
recursive 
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data:  var  key:RecKey;  group(Tkey);  groupl ist ( tk ey)  . 

grouplist  (foldkey: RecKey) :  var  key:RecKey; 

group(Tkey) ;  old key#key;~grouplist (tkey) 

□  §• 

group  (Tkey):  var  key:RecKey; 

record (|key)  ;  reclist  (tkey)  . 

reclist (tkey : RecKey) :  var  newkey : RecKey ; 

record (Tnewkey) ;  key=newkey;  reclist (tkey) 

D  §• 

The  iterative  formulation  is: 

data:  var  key  ,oldkey : RecKey ; 
group  (T  old  key)  ; 

*(  group(Tkey);  oldkey#key;  oldkey:=key  )  . 

group  (Tkey) :  var  key , newkey : RecKey ; 

record  (Tkey)  ;  *  (  record  (Tnewkey)  ;  key=newkey  ). 

This  example  demonstrates  a  number  of  advantages  of 
agp's.  First,  it  shows  how  a  formal  agp  structure  can  be 
designed  to  mirror  the  viewed  logical  structure  of  the 
language.  This  is  especially  true  in  the  iterative 
formulation,  which  shows  how  iteration  can  be  used  to 
replace  recursion  in  an  agp.  The  conciseness  of  this 
solution  is  particularly  worth  noting.  The  insertion  of 
processing  actions  within  this  program  skeleton  is  now  a 
straightforward  matter.  Also,  by  the  simple  change  of 
"oldkey*key"  to  H oldkey<keyH ,  we  describe  a  sorted  record- 
group  language. 

6.3  Parts  Movement  Problem 

The  next  example  is  an  agp  solution  to  the  parts 
movement  problem  presented  in  Section  3.2,  which  served  as 
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an  example  for  the  Jackson  Method.  Recall  that  the  problem 
is  as  follows.  A  factory  warehouse  receives  and  issues 
parts.  For  each  transaction  there  is  a  data  record  with 
three  fields:  the  part  number,  the  transaction  code 

(•issue1  or  'receipt*),  and  the  quantity.  The  data  records 
are  available  in  sorted  order  by  part  number.  The  task  is 
to  produce  a  written  summary  of  the  net  movement  for  each 
part. 

In  the  following  agp  solution,  we  assume  that  each  part 
number  is  of  data  type  PartNumber,  each  transaction  code  of 
type  Transcode,  each  quantity  of  type  Quantity  (a  subrange 
of  integer) ,  and  each  net  of  type  Net  (also  a  subrange  of 
integer).  The  nonterminal  "rec"  is  a  token  describing  a 
data  record.  Its  three  output  affix-parameters  contain  the 
described  data  record's  part  number,  transaction  code,  and 
quantity,  respectively.  Also,  we  assume  "d isplay  (pn, n) " 
produces  a  listing  of  the  net  movement  "n”  for  part  ”pnM. 
data:  var  oldpn, pn: Part  Number ,  net:Net; 

group  (Toldpn, net)  ;  display  (foldpn,net)  ; 

*(  group  (Tpn,  net)  ;  oldpn<pn;  displa y ( tpn, net) ; 
oldpn: =pn  )  . 

group  (Tpn, net ) : 

var  pn,newpn: PartNumber ,  tc:TransCode,  q: Quantity, 
net : Net ; 

rec  (Tpn  ,tc,q);  update  (fO,  tc  ,q ;  Tnet)  ; 

*(  rec ( Tnewpn, t c, q) ;  pn=newpn; 
update  (fnet ,  tc ,  q  ;Tnet)  ). 

update  (t  net:Net,  tc : Transcode,  q:Quantity;  Tnewnet) : 
var  ne wnet: Net ; 

(  tc='issue';  ne  wnet :  =  ne  t  -  q 
[J  tc= '  receipt '  ;  newnet :  =net+q  ). 
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The  structure  of  the  data  in  this  problem,  it  should  be 
noted,  is  the  same  as  that  for  the  sorted  record-group 
language.  We  see  how  easily  the  processing  specific  to  the 
parts  movement  problem  was  placed  within  the  record-group 
skeleton  to  produce  an  agp  program.  Once  a  proper  structure 
was  designed,  the  rest  followed  almost  immediately. 

This  example  also  shows  how  data  and  processing  can  be 
specified  together  in  a  unified  manner.  In  particular,  the 
"update"  grammar  rule  is  pure  computation,  while  the  others 
combine  computation  and  data.  The  conciseness  and 
modularity  of  this  solution  is  particularly  noteworthy  when 
compared  with  Jackson's  solution. 

Another  point  worth  noting  is  the  use  of  "net"  as  both 
an  input  and  an  output  parameter  of  "update"  in  the  "group" 
grammar  rule.  Since  we  think  of  input  parameters  as  being 
passed  by  value  and  output  parameters  by  result  and  there 
are  no  global  variables,  there  are  no  aliasing  problems. 
This  also  shows  how  a  variable  can  be  redefined. 

6.4  A  Te x t  Processing  Problem 

The  next  example  is  a  problem  posed  by  Dijkstra  [72]. 
The  problem,  as  given  by  Dijkstra,  is  as  follows: 
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Problem: 

We  consider  a  character  set  consisting  of  letters, 
a  space  (sp)  ,  and  a  point  (pnt)  .  Words  consist  of  one 
or  more,  but  at  most  twenty  letters.  An  input  text 
consists  of  one  or  more  words,  separated  from  each  other 
by  one  or  more  spaces  and  terminated  by  zero  or  more 
spaces  followed  by  a  point.  With  the  character  valued 
function  PNC  (Read  Next  Character)  the  input  should  be 
read  from  and  including  the  first  letter  of  the  first 
word  up  to  and  including  the  terminating  point.  An 
output  text  has  to  be  produced  using  the  primitive 
PNC  (x )  (Print  Next  Character)  with  a  character  valued 
parameter.  The  text  is  to  be  subjected  to  the  following 
transformation: 

(1)  in  the  output  text,  successive  words  have  to  be 
separated  by  a  single  space; 

(2)  in  the  output  text,  the  last  word  has  to  be  followed 
by  a  single  point; 

(3)  when  we  number  the  words  0,1, 2, 3,...  in  order  from 
left  to  right,  the  words  with  an  even  ordinal  number 
have  to  be  copied,  while  the  letters  of  the  words 
with  an  odd  ordinal  have  to  be  printed  in  reverse 
order . 


Dijkstra's  solution  directly  manipulates  a  word’s  data 
representation,  a  character  array.  In  light  of  the  recent 
research  regarding  data  abstractions,  a  word  data  type  is 
more  appropriate.  The  relevant  operations  are: 

( 1 )  1  1  -  null  word 

/ 

(2)  s||c  -  character  c  appended  to  word  s 

(3)  s[i]  -  i*th  character  of  word  s 

(4)  len(s)  -  length  of  word  s 

The  first  solution  presented  is  exactly  like  Dijkstra's 
except  that  all  word  manipulations  are  in  terms  of  the  word 
data  type  operations. 
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Boolean  forward;  char  x;  word  s; 
integer  k#inc,term7” 

f orward:=true ;  x:=RNC; 

repeat 

c«  ~ 1  I  • 
o .  » 

repeat  s:=s|jx;  x:=RNC  until  x=sp  or  x=pnt; 
while  x=sp  do  x:=RNC; 
if  forward 

then  begin  k:=0;  inc:=+1;  term:  =len  (s)  end 
else  begin  k: =len  (s) + 1 ;  inc:=-1;  term: = 1  end; 

repeat  k:=k+inc;  PNC(s[k])  until  k=ter m; 

if  x=pnt  then  PNC(pnt)  else  PNC(sp); 

f or war d : =n on  forward 

until  x=pnt 

Two  design  decisions  in  the  above  program,  which  we 
shall  adopt  without  criticism,  are:  (1)  the  code  to  print 
forward  and  the  code  to  print  backwards  share  a  single  print 
loop  by  having  each  case  specify  the  index  of  the  first 
letter,  the  index  of  the  last  letter,  and  the  direction  (as 
an  increment)  in  the  variables  k,  term,  and  inc, 
respectively;  (2)  the  print  loop  is  an  iterative,  rather 
than  recursive,  formulation.  Other  solutions  that  do  not 
adopt  these  decisions  could  easily  be  written,  but  are 
omitted  since  they  do  not  illustrate  any  new  points. 
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An  rcfg  grammar  for  the  input  language  is: 


text:  word;  *  (  +  sp;  word  ) ;  *sp;  pnt. 
word:  ^letter, 
sp:  »  ». 

pnt:  •  .  • . 

The  agp  grammar  rules  are: 

text:  yar  forward: Boolean,  s:Word; 

forward : =tr ue ; 

word(Ts);  put  (fs,  forward)  ; 

*(  +sp;  word(Ts);  print  ('  '); 

f orward:  =non  (forward)  ;  put  (ts,  forward)  ); 

*sp;  pnt;  print  (*.*)• 

word(ts):  var  s:Word,  ch :  char  ; 

s :  = 1  1  ;  +  (  letter(|ch);  s:  =  s||ch  ). 

put  (fs: Word,  f :  Boolean)  : 

var  k, terra, inc: integer ; 

(  forward;  k:  =  1;  term:  =  len  (s)  ♦  1 ;  inc:=+1 
G  non(forward)  ;  k:  =  len(s);  term:  =  0;  inc:  =  -1  ); 

*(  k^term;  print(s[k]);  k:=k  +  inc  ). 

sp:  •  *. 

pnt:  •  .  * . 

Again,  we  see  that  once  a  proper  data  structure  is 
developed,  the  program  follows  easily.  The  agp  formalism 
has  all  the  power  of  a  conventional  programming  language  but 
is  more  descriptive.  An  agp  program  describes  in  a  clear 
and  concise  manner  how  the  data  is  structured  and  how  it  is 
to  be  processed.  In  a  conventional  programming  language, 
this  structure  is  not  even  specified;  one  must  attempt  to 
infer  it  from  the  program. 
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6 •  5  The  Telegram  Problem 


This  last  example  is  a  pro 
to  process  a  stream  of  telegrams 
many  critiques  and  alternative 
including  Jackson  [75],  McKeem 
Gries  [77],  Many  of  these  effo 
of  precision  and  completeness  in 
which  follows. 

A  program  is  require 
telegrams.  This  stream  is 
letters,  digits,  and  blanks 
transferred  in  sections  of 
buffer  area  where  it  is  to 
the  telegrams  are  separated 
each  telegram  is  delimited 
stream  is  terminated  by  th 
telegram,  that  is  a  teleg 
telegram  is  to  be  processed 
chargeable  words  and  to 
overlength  words.  The  word 
chargeable  and  words  of  mo 
length  are  considered  over 
processing  is  to  be  a  neat 
each  accompanied  by  the 
indicating  the  occurrence  of 


blem  posed  by  Henderson  [72] 
.  The  problem  has  attracted 
solutions  in  the  literature, 
an  [76],  Noonan  [75],  and 
rts  have  addressed  the  lack 


the 

origina  1 

spe 

cif  ic 

ati 

on , 

d  to 

process 

a 

stre 

am 

of 

available  as 

a 

seque 

nee 

of 

on 

some  dev 

ice 

and 

can 

be 

predetermine 

d  s 

ize  i 

nto 

a 

be  processed. 

The  wo 

rds 

in 

by 

sequences 

of 

blan 

ks 

and 

by 

the  wor 

d  "Z 

ZZZ" . 

The 

e  occurrence 

of 

the 

em 

pty 

ram 

with  no 

wo 

rds. 

Each 

to  determine  the  number  of 
check  for  occurrences  of 
s  "ZZZZ"  and  "STOP”  are  not 
re  than  twelve  letters  in 
length.  The  result  of  the 
listing  of  the  telegrams, 
word  count  and  a  message 
an  overlength  word. 


There  are  a  few  ambiguities 
we  clear  up  informally.  First 
input  stream  may  well  break  ac 
must  remain  intact  during  proce 
in  fact,  makes  two  words  out  of 
is  possible  for  the  input  strea 
telegram.  Any  input  line,  incl 
with  a  nonempty  sequence  of  bl 
that  the  input  terminates  with 


in 

this  speci 

fication 

that 

note  that  a 

word  in 

the 

OSS 

a  buffer 

boundary 

yet 

ssing.  Henderson's  program, 
each  such  broken  word.  It 
m  to  contain  only  the  empty 
uding  the  first,  may  begin 
anks.  In  addition,  we  note 
a  telegram  with  no  words. 


-89- 


Henderson's  program,  in  contrast,  terminates  on  a  telegram 
with  no  chargeable  words. 

The  specification  does  not  say  how  overlength  words  are 
to  be  printed.  For  now,  we  assume  that  only  the  first 
twelve  characters  of  an  overlength  word  are  printed.  A 
discussion  of  this  assumption  and  its  alternative  (i.e., 
printing  overlength  words  in  their  entirety)  appears  near 
the  end  of  the  section. 

As  we  examine  the  structure  of  telegrams  and  compare  it 
with  the  structure  of  the  input,  we  see  that  we  have  a 
classical  structure  clash  on  our  hands.  We  view  the  data  as 
a  sequence  of  telegrams  when  it  really  consists  of  a 
sequence  of  buffers,  and  there  is  no  particular  relationship 
between  telegrams  and  buffers.  A  telegram  may  begin  and  end 
anywhere  in  a  buffer  and  may  span  several  buffers.  Since  we 
view  a  telegram  as  a  sequence  of  words,  words  are  a  natural 
choice  for  a  token  class.  However,  this  does  not  resolve 
the  clash  since  a  word  may  span  input  buffers.  To  resolve 
this  structure  clash  we  need  to  go  to  a  unit  that  is  small 
enough  to  be  common  to  both  structures.  The  largest  such 
unit  is  the  character.  Thus,  we  use  three  modules.  The 
first,  a  built-in  module,  takes  input  buffers  and  produces 
characters.  The  second  takes  characters  and  produces  words. 
The  third  takes  words,  structures  them  into  telegrams,  and 
produces  telegram  reports. 

Our  decision  to  print  only  the  first  twelve  letters  of 
an  overlength  word  implies  that  only  the  first  twelve 
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letters  of  any  word  need  be  stored.  This  decision  is 

reflected  in  the  Word  data  type  operations.  Given  in  terms 

of  a  PL/I  character  type,  they  are: 

str:  Word  ->  Character ( 1 2)  Varying 

len:  Word  ->  Integer 

overlen:  Word  ->  Boolean 
append:  Word  x  Char(1)  ->  Word 

null:  ->  Word. 

If  w  represents  a  non-overlength  word,  then  str  (w)  is  the 
character  string  of  the  represented  word,  overlen (w)  = 

false,  and  len(w)  is  the  length  of  str(w).  If  w  represents 
an  overlength  word,  then  str (w)  is  the  first  12  characters 
of  the  represented  word,  overlen (w)  =  true,  and  len  (w)  =  12. 

Append  is  defined  such  that  if  len (w)  <  12  then 
len  (append  (w ,  a)  )  =  len  (w) +1  and  over  len  (append  (w ,  a) )  = 

false ;  if  len  (w)  =  12  then  len  (append  (w , a)  )  =  12  and 

overlen  (append  (w,  a)  )  =  true. 

The  following  input  grammar  describes  the  structure  of 
a  stream  of  telegrams,  using  "word”  as  a  token. 

telestream:  ♦telegram;  endt. 

telegram:  wstream;  endt. 

wstream:  t-teleword. 

teleword:  var  w:Word; 

word (T»)  ;  str (w)  *  * ZZZZ * . 

endt:  var  w: Word ; 

word (T*);  str (w) =*  ZZZZ* . 
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The  resulting  set  of  grammar  rules  for  the  third  module 


is: 


telestream:  println (• TELEGRAM  ANALYSIS'); 
♦telegram;  endt. 

telegram:  var  #wd,  #ol:integer; 

println ( ' ' ) ; 

wstream (T#wd, #ol ) ;  endt; 
println  ( 'WORD  COUNT:  »,  #wd)  ; 
println ( 'OVERLENGTH  COUNT: ' , #ol) . 


wstream (T#wd,  #ol) : 

v§r  #wd , to 1: integer ,  w:Word; 

#wd: =0;  #ol:=0; 

+  (  teleword  (Tw)  ; 

(  str  (w)  *  '  STO  P'  ;  #wd:=#wd+1 
0  str  (w)  =  *  STO  P*  )  ; 

(  overlen (w) ;  #ol:=#ol+1 
D  fion  (overlen  (w)  )  )  ; 
print  (str  (w)  )  )  . 

teleword  (T w)  : 

var  w : Wo rd ; 

word  (Tw)  ;  str  (w)  * ' ZZZZ ' . 

endt:  var  w:Word; 

word  (Tw)  ;  str (w) =' ZZZZ « . 


The  grammar  rules  for  the  second  module  are: 

input:  var  w:Word; 

♦  sp;  word(Tw);  e mit  ( word  (w)  )  ; 

♦  (  +  sp;  word(Tw)  ;  emit  (word  (w)  )  )  . 

word (Tw) :  var  w:Word,  ch:char; 
w :=null ; 

♦  (  (letter  (Tch)  □  digit  (Tch)  )  ;  w:  =  append  (w ,ch)  ). 

sp:  '  ' . 

Since  there  is  no  clash  between  words  and  telegrams,  as 
there  is  between  buffers  and  words,  it  is  not  necessary  to 
keep  modules  2  and  3  separate.  In  this  case,  the 
consolidated  module  would  have  to  filter  words  from  the 
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character  stream  itself.  These  considerations  make  its 


structure  somewhat  awkward  and,  as  a  result,  the  resulting 
solution  is  not  as  "clean"  as  the  one  where  the  modules  are 
separate.  We  should  not  be  surprised  at  this  since  our  view 
of  telegrams  is  stated  in  terms  of  a  word  stream  and  not  a 
character  stream .  As  we  see  in  module  two,  the  filtering  of 
words  from  the  character  stream  is  somewhat  awkward,  and  we 
would  expect  this  awkwardness  to  increase  if  more  structure 
were  imposed  on  the  character  stream. 

The  above  program  has  two  nice  aspects  as  a  result  of 
our  decision  to  print  only  the  first  twelve  letters  of  an 
overlength  word:  the  implementation  of  the  Word  data  type 
is  very  simple  since  word  lengths  are  bounded,  and  words  can 
be  printed  on  a  word-by-word  basis.  This  second  aspect 
means  it  is  very  easy  to  modify  the  program  so  that  it  does 
not  print  out  certain  words.  If  the  decision  were  made  to 
print  overlength  words  in  their  entirety,  one  of  these 
aspects  would  have  to  be  sacrificed.  If  words  were  still  to 
be  printed  on  a  word-by-word  basis,  the  operations  on  the 
Word  data  type  must  be  modified  so  that  they  can  describe, 
in  their  entirety,  words  of  arbitrary  length.  This  requires 
a  rather  complicated  type  implementation.  On  the  other 
hand,  words  could  be  printed  on  a  character-by-character 
basis  from  within  module  2.  This  way  only  an  indication  of 
whether  a  printed  word  was  overlength  need  be  kept;  the 
word  need  not  be  stored.  However,  this  method  is 
less  attractive  if  modifications  to  the  specifications  in 
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terns  of  words  were  made. 


Noonan,  in  his  paper  "Structured  Programming 
Specification"  [75],  derives  what  he  calls 
specification  for  the  Telegram  Problem.  It  is 
to  note  that  his  formal  specification  is  an 
grammar  very  similar  to  our  agp  program! 


and  Formal 
a  formal 
interesting 
attribute 
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CHAPTER  2  ~  FURTHER  TOPICS 


In  this  chapter  we  touch  on  a  few  points  about  agp's 
that  do  not  fit  into  the  exposition  of  the  previous  two 
chapters. 

7 . 1  Hultgple  Serial  Input  Streams 

As  presented,  an  agp  can  handle  only  one  serial  input 
stream.  A  very  simple  extension  allows  n  serial  input 
streams  to  be  handled.  This  is  done  by  partitioning  the  set 
of  token  classes  into  n  subsets,  one  subset  for  each  input 
stream.  Under  this  scheme,  an  agp  not  only  describes  each 
of  the  n  streams  individually,  but  also  specifies  how  the 
streams  are  collated. 

7 . 2  Error  Handling 

If  a  certain  set  of  inputs  are  regarded  as  "bad  data" 
and  are  to  be  handled  in  a  special  way,  then  this  set  and 
the  processing  of  its  elements  must  also  be  specified  in  the 
agp.  Thus,  the  set  of  accepted  inputs  contains  both  "good 
data"  and  "bad  data". 

Inputs  not  in  the  accepted  set  of  an  agp  are  rejected 
by  the  executing  agp  program  as  "invalid".  In  grammatical 
terms,  the  parser  has  determined  that  the  input  is  not  in 
the  language  described  by  the  grammar  (program)  and  rejected 
the  input.  This  leads  to  the  important  result  that  there 
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are  no  hidden  special  cases  in  an  agp.  The  processing  is 
well-defined  for  every  possible  input.  If  the  set  of 
accepted  inputs  includes  all  possible  inputs,  then  no  inputs 
are  "invalid”. 

A  topic  for  future  research  is  the  investigation  of  an 
escape  alternative  such  as  "else  §"  that  can  be  used  as  an 
alternative  in  a  regular  expression.  Instances  of  a  would 
be  used  only  if  no  other  alternatives  could  be  applied.  The 
"else"  alternative  scheme  would  make  it  easier  to  specify 
complement  sets.  This  makes  it  particularly  applicable  to 
error  processing,  though  it  is  not  specifically  an  error 
handling  mechanism. 

7 . 3  Nondeterminism 

Agp*s,  in  their  full  generality,  provide  a  general 
framework  for  nondeterministic  programming.  A  major  topic 
for  future  research  is  the  development  of  sufficient 

conditions  for  deterministic  execution. 

/ 

Given  a  deterministic  head  grammar  for  an  agp,  the  only 
source  of  possible  nondeterminism  is  predicates.  A  possible 
approach,  then,  is  to  start  with  Watt’s  AF-LR  parser  and 
enhance  it  to  allow  predicates  to  influence  the  flow  of 
control.  Conditions  need  be  developed  to  guarantee 
deterministic  flow.  Some  work  in  this  area  has  been  done  by 
Watt  [74,  77b],  A  necessary  condition  for  determinism  is 
that  "parallel"  predicates  be  exclusive.  That  is,  if 
"(pi  bl  Q  p2  b2)  "  is  a  subexpression  in  a  grammar  rule  and 
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pi,  p2  are  predicates,  then  pi  and  p2  can  never  both  be 


true. 


7.4  Imple»entation 

The  topic  of  agp  implementation  is  closely  tied  to  the 
previous  topic  of  nondeterminism  and  is  the  most  important 
area  for  future  research.  In  Chapters  2  and  4  we  devoted 
attention  to  parsing  to  illuminate  some  implementation 
problems  and  possible  approaches  to  their  solution.  Very 
much  work,  however,  is  needed  in  this  area,  first  to 
identify  the  problems  and  then  to  develop  solutions.  This 
work  must  be  done  for  agp's  to  be  more  just  a  very  useful 
but  unimplemented  (or  uni mple men table)  design  tool. 
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CHAPTER  8  -  SUMMARY  AND  CONCLUSIONS 

In  this  thesis  we  have  developed  a  formalism  that 
synthesizes  the  specification  of  both  data  and  algorithms. 

Many  of  the  seminal  ideas  for  our  approach  came  from 
the  Jackson  Method,  which  shows  how  to  construct  a  program 
based  on  the  structure  of  the  data  it  processes.  Jackson’s 
design  approach  is  quite  attractive  and  serves  as  the  basis 
for  our  own  design  efforts.  There  are,  however,  problems 
with  the  way  the  design  approach  is  developed  into  a  design 
method . 

The  fundamental  problem  is  that  design  concerns  in 
Jackson's  method  are  too  closely  tied  to  purely 
implementation-dependent  concerns.  Since  Jackson  uses  an 
rcfg-like  formalism  as  a  structuring  device  only,  and  not  as 
a  generative  grammar,  the  program  must  explicitly  read  and 

parse  the  input  data.  As  a  result,  parsing  concerns  must  be 

\ 

incorporated  into  the  program  design.  These  parsing 
concerns,  however,  are  not  inherent  in  the  program  design 
and  serve  only  to  cloud  its  structure. 

Our  approach  is  based  on  separating  design  and 
implementation-dependent  concerns.  Rather  than  just  have 
the  program  use  grammar-like  structuring  mechanisms,  we  want 
it  to  be  an  actual  grammar. 

With  a  grammatical  approach,  the  program  structures  the 
data  and  specifies  how  the  data  are  to  be  processed.  It 


-98- 


does  not  parse  the  data:  the  structure  of  the  program 
implicitly  specifies  a  parser,  which  is  supplied  by  a  parser 
generator  system.  The  parser  generator  system  "compiles" 
the  program  into  an  executable  version.  Without  parsing 
considerations,  the  program  structure  is  more  transparent 
and  higher  level. 

This  approach  allows  an  elegant  separation  of  design 
concerns  from  implementation  concerns.  The  designer  can 
concentrate  on  his  primary  concern,  the  creative  process  of 
designing  a  proper  program  structure  and  placing  operations 
within  it.  The  secondary  concern,  the  automatable  process 
of  making  the  design  executable,  is  localized  within  the 
compiler. 

As  a  result  of  this  division  of  labor,  each  component 
of  the  complete  system  can  be  specialized  to  perform  its 
corresponding  task  more  effectively.  From  the  design  side, 
this  means  a  formalism  can  be  chosen  that  is  best  able  to 
specify  and  add  meaning  to  structure.  From  the  parsing 
side,  this  opens  the  door  to  a  wide  variety  of  sophisticated 
parsing  algorithms  and  optimizations  that  in  practice  are 
too  complicated  or  tedious  to  be  implemented  by  an 
individual  programmer. 

A  suitable  grammatical  formalism  must  do  more  than  just 
link  processing  to  data.  Consistency  demands  that  the 
specification  of  data  and  computations  be  put  in  a  unified 
framework.  The  basis  for  a  unified  framework  is  the  fact 
that  the  same  structuring  mechanisms  can  be  used  for  both 
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data  and  algorithms.  Otherwise,  data  and  computations 
require  separate,  though  fundamentally  similar,  schemes. 

The  formalism  must  also  provide  parameterized 
interfaces  so  that  the  flow  of  data  is  visible  and 
controlled.  The  immediate  consequence  of  parameterization 
is  that  grammar  rules  are  decoupled,  inducing  a  high  degree 
of  independence  among  rules  in  a  grammar  and  among  modules 
in  a  system.  From  modularity  comes  "local  scoping”  within 
each  rule.  Together,  these  effects  mean  that  processing  is 
able  to  take  full  advantage  of  syntactic  structure.  Thus, 
processing  is  not  just  tacked  onto  the  syntax  description, 
but  rather  is  incorporated  as  an  integral  part. 

We  also  want  the  formalism  to  have  regularized  rules. 
Regularized  rules  permit  us  to  specify  a  formal  structure 
that  coincides  with  the  viewed  logical  structure.  Their 
higher  level  constructs  make  the  logical  structure  more 
transparent  and  improve  both  readability  and  wr iteability . 

In  looking  at  previously  proposed  enhanced  grammars,  we 

\ 

saw  that  each  was  deficient  in  two  of  the  three  above 
respects.  In  response,  we  developed  a  grammatical  formalism 
that  satisfies  all  of  the  above  criteria.  This  formalism 
has  all  the  power  of  a  conventional  programming  language, 
but  is  more  concise  and  descriptive.  A  grammar  in  the 
formalism  is  called  an  agp. 

An  agp  is  at  the  same  time  a  program  and  a  grammar. 
The  dual  nature  of  agp's  comes  into  play  in  programs  that 
involve  input  data,  where  an  agp  specifies  in  a  clear  and 
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concise  manner  the  precise  set  of  inputs  accepted,  how  they 
are  structured,  and  how  they  are  processed.  The  execution 
of  an  agp  is  data-driven.  For  a  computation-oriented 
problem,  in  which  little  input  data  is  involved,  an  agp 
solution  is  much  like  a  solution  in  a  conventional 
programming  language.  Agp's  distinguish  themselves  in  more 
data-oriented  problems,  where  the  grammatical  basis  can  be 
used  to  advantage.  For  these  problems,  as  we  saw  in  the 
examples  given,  once  a  proper  data  structure  is  developed, 
the  rest  of  the  program  follows  almost  immediately. 

A  further  advantage  of  this  grammatical  formalism  is 
that  there  are  no  hidden  special  cases  in  an  agp.  The 
processing  is  well-defined  for  all  possible  inputs, 
including  those  not  in  the  accepted  set  of  inputs. 

The  result  of  this  grammatical  approach  to  programming 
is  thus  a  more  declarative  program  -  one  that  is  easier  to 
write,  easier  to  read,  and  more  likely  to  be  correct. 
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